💻 computer science

Content-Aware Mamba for Learned Image Compression

This paper introduces Content-Aware Mamba (CAM), a novel state-space model that dynamically adapts token processing and injects global priors to overcome the rigidity of standard Mamba scans, enabling a learned image compression framework (CMIC) that achieves state-of-the-art rate-distortion performance surpassing VTM-21.0 by significant margins.

Original authors: Yunuo Chen, Zezheng Lyu, Bing He, Hongwei Hu, Qi Wang, Yuan Tian, Li Song, Wenjun Zhang, Guo Lu

Published 2026-03-12

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yunuo Chen, Zezheng Lyu, Bing He, Hongwei Hu, Qi Wang, Yuan Tian, Li Song, Wenjun Zhang, Guo Lu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Smart Filing System"

Imagine you have a massive, messy library of photos (your image). Your goal is to shrink this library down to fit in a tiny backpack (compression) without losing the story of the photos.

Traditional methods (like the old VTM standard) are like a librarian who files books strictly by their position on the shelf: "Book 1, then Book 2, then Book 3." They don't care if Book 1 is a picture of a cat and Book 50 is also a picture of a cat. They just file them in order. This is inefficient because the librarian misses the chance to say, "Hey, these two are the same! Let's just write 'Cat' once and skip the rest."

Mamba is a newer, faster librarian who can read the whole library much quicker than the old one. However, the standard Mamba still has a flaw: it reads the books in a strict "snake-like" pattern (left-to-right, top-to-bottom). It's fast, but it's content-agnostic. It doesn't look at what the book is about; it just looks at where it is.

This paper introduces CMIC (Content-Aware Mamba), a super-smart librarian who breaks the rules to save space. It does two magical things:

1. The "Re-Ordering" Trick (Content-Adaptive Token Permutation)

The Problem:
Imagine you are packing a suitcase. You have 50 red socks scattered all over the floor, mixed in with 50 blue socks. If you follow a strict "pick up everything in a grid pattern" rule, you might pick up a red sock, then a blue one, then a red one, then a blue one. You can't pack them efficiently because they are mixed up.

The Solution:
CMIC looks at the socks and says, "Wait a minute! Let's grab all the red socks first, then all the blue socks." It physically rearranges the order of the items before packing them.

How it works: Instead of scanning the image from top-left to bottom-right, the AI looks at the image content. It finds all the "clouds," groups them together. It finds all the "tree leaves," groups them together.
The Result: By grouping similar things together, the AI can say, "Oh, I've seen this cloud pattern before; I don't need to write it down again." This eliminates redundancy much better than a rigid grid scan.

2. The "Cheat Sheet" (Global-Prior Prompting)

The Problem:
Mamba is a "causal" model. Think of it like a person reading a book who can only see the words they have already read. They cannot see the words coming up next.
In an image, this is weird. If you are looking at a person's left eye, the right eye is "future" information in a left-to-right scan. A strict Mamba model ignores the right eye until it gets there, missing the context that "this is a face."

Usually, to fix this, models try to scan the image from all four directions (top, bottom, left, right) at once. But that's like having four librarians reading the same book simultaneously—it's too slow and uses too much energy.

The Solution:
CMIC gives the librarian a Cheat Sheet (a "Prompt") before they start reading.

This cheat sheet is a summary of the entire image. It says, "By the way, this image is mostly blue sky with a few birds."
Even though the librarian is still reading left-to-right, the cheat sheet allows them to "know" about the birds and the sky immediately.
The Result: The model gets the benefit of seeing the whole picture (global context) without needing four librarians. It keeps the speed of one librarian but the wisdom of a team.

Why is this a Big Deal?

The authors built a system called CMIC using these two tricks. They tested it against the current best compression standards (like VTM-21.0) and other AI compressors.

The Score: CMIC saved 15% to 21% more space than the current best standard while keeping the picture quality exactly the same.
The Analogy: If the old standard could fit 100 photos in a backpack, CMIC can fit 120 photos in the same backpack with the same clarity.
Speed: Despite being smarter, it didn't get slower. It's still as fast as the other Mamba models.

Summary in One Sentence

The paper teaches an AI how to stop reading an image like a rigid robot (scanning line-by-line) and start reading it like a human (grouping similar things together and using a mental summary of the whole scene) to shrink files much more efficiently.

1. Problem Statement

The paper addresses the limitations of applying Mamba-style State-Space Models (SSMs) to Learned Image Compression (LIC). While Mamba offers global receptive fields with linear computational complexity (unlike the quadratic complexity of Transformers), standard Mamba implementations face two critical issues when applied to 2D images:

Content-Agnostic Scanning: Standard Mamba relies on a fixed, predefined raster scan (or multi-directional scans). This rigid order processes tokens based on spatial proximity rather than feature similarity. Consequently, tokens that are semantically related but spatially distant are separated in the sequence, while unrelated tokens are grouped together. This hinders the model's ability to effectively eliminate redundancy between content-correlated regions.
Strict Causality: Mamba is inherently a causal sequence model, meaning a token can only attend to preceding tokens in the scan order. Images are non-causal 2D structures. While multi-directional scanning attempts to mitigate this, it quadruples computational complexity and still relies on Euclidean spatial paths, failing to capture true global semantic dependencies efficiently.

2. Methodology: Content-Aware Mamba (CAM)

The authors propose Content-Aware Mamba (CAM), a novel SSM architecture designed to dynamically adapt its processing to image content. The core of the method is the CMIC (Content-Aware Mamba-based Image Compression) model, which integrates two key mechanisms:

A. Content-Adaptive Token Permutation (CTP)

Instead of processing tokens in a fixed raster order, CAM reorganizes the token sequence based on feature similarity.

Codebook-Based Clustering: To avoid the computational cost and instability of online K-Means, the authors employ a shared, learnable codebook (inspired by VQ-VAE). Each CAM block maintains its own codebook of centroids updated via an Exponential Moving Average (EMA) of K-Means assignments during training.
Permutation Strategy: Tokens are assigned to clusters based on cosine similarity to these centroids. The token sequence is then permuted so that tokens belonging to the same cluster (i.e., content-similar tokens) become contiguous in the 1D sequence.
Effect: This allows the SSM to prioritize interactions between semantically related tokens regardless of their original spatial distance, effectively capturing long-range redundancy in feature space rather than Euclidean space.

B. Global-Prior Prompting (GPP)

To mitigate the strict causality of SSMs without resorting to expensive multi-directional scans, the authors introduce a prompting mechanism.

Redundancy-Aware Dictionary: A learnable dictionary maps cluster centroids to prompt vectors.
Sample-Specific Prompts: For a given image, a one-hot assignment matrix is generated based on the clustering results. This queries the dictionary to create a sample-specific prompt matrix ( $P$ ) that encodes global statistics and redundancy distribution.
State Modulation: These prompts are injected into the output projection matrix ( $C$ ) of the SSM state-space equation ( $O_i = (C + P)h_i + Dx_i$ ).
Effect: This allows information from the entire image (global priors) to influence the state update at every step, effectively relaxing the causal constraint and enabling the model to "see" beyond the immediate scan sequence.

3. Key Contributions

Content-Adaptive Token Permutation: A novel mechanism that reorders the scan sequence based on feature similarity using a stable, codebook-based clustering strategy. This shifts the focus from spatial adjacency to feature-space proximity.
Global-Prior Prompting: A method to inject sample-specific global statistics into the SSM via a redundancy-aware dictionary. This mitigates strict causality and enhances global modeling capabilities with minimal computational overhead.
CMIC Model: An end-to-end learned image compression framework that integrates CAM blocks into a standard VAE-based architecture. It achieves state-of-the-art performance while maintaining linear complexity.

4. Experimental Results

The CMIC model was evaluated on three standard datasets: Kodak, Tecnick, and CLIC.

Rate-Distortion Performance:
- CMIC significantly outperforms the traditional codec VTM-21.0 (Versatile Video Coding reference software), achieving BD-rate savings of 15.91% (Kodak), 21.34% (Tecnick), and 17.58% (CLIC).
- It surpasses previous SOTA Mamba-based LIC models (MambaVC and MambaIC) by a significant margin (e.g., ~7.5% to 10% BD-rate improvement over MambaVC).
- It also outperforms Transformer-based and hybrid CNN-Transformer models (e.g., FTIC, TCM-L) in both PSNR and MS-SSIM metrics.
Efficiency and Complexity:
- Computational Cost: Despite the added mechanisms, CMIC maintains linear complexity. It reduces FLOPs by 36% compared to TCM-L and 57% compared to MambaIC.
- Memory & Latency: It achieves a 78% reduction in GPU memory usage compared to MambaIC (attributed to avoiding quadratic 2D scans) and maintains competitive decoding latency (0.405s for 2K images).
Ablation Studies:
- Removing either CTP or GPP results in performance drops, confirming their complementary roles.
- Visualizations of the Effective Receptive Field (ERF) show that CMIC captures content-adaptive, semantic regions (e.g., feathers, clouds) rather than the isotropic or cross-shaped patterns seen in other models.

5. Significance

This work represents a significant advancement in Learned Image Compression by successfully adapting State-Space Models to the specific structural requirements of images.

Bridging the Gap: It solves the fundamental mismatch between the 1D causal nature of Mamba and the 2D non-causal, content-correlated nature of images.
Efficiency vs. Performance: It demonstrates that high-performance compression does not require the quadratic complexity of Transformers or the heavy computational cost of multi-directional scanning.
Semantic Awareness: By introducing content-aware scanning and global prompting, the model moves beyond spatial redundancy to capture semantic redundancy, which is crucial for high-quality compression at low bitrates.

In conclusion, CMIC establishes a new state-of-the-art for learned image compression, offering a highly efficient, content-adaptive alternative to both Transformer-based and standard Mamba-based approaches.