💻 computer science

Liquid Fusion of Heterogeneous Representations Towards General Salient Object Detection

The paper proposes LFNet, a novel framework that leverages a liquid fusion mechanism to dynamically integrate complementary spectral representations from CNN and State Space Model backbones, achieving state-of-the-art performance across diverse general salient object detection tasks.

Original authors: Ke Chen, Ling Zhou, Guangqi Jiang, Gengshen Wu, Yi Liu, Shoukun Xu

Published 2026-06-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ke Chen, Ling Zhou, Guangqi Jiang, Gengshen Wu, Yi Liu, Shoukun Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to find a specific, shiny toy hidden in a messy room. To do this well, you need two different ways of looking at the world:

The "Big Picture" Gazer: This person steps back and looks at the whole room to understand the general layout and where things are likely to be. They are great at seeing the flow and the big shapes but might miss tiny details like a scratch on the toy.
The "Detail" Detective: This person gets right up close. They are amazing at spotting tiny edges, textures, and sharp lines, but they might get so focused on the small stuff that they lose track of where the toy actually fits in the room.

For a long time, computer scientists tried to build a single "super-eye" that could do both jobs perfectly. They tried to make the "Big Picture" Gazer look closer or the "Detail" Detective step back, but it turns out these two styles of thinking are fundamentally different. One is like listening to a smooth, continuous melody (State Space Models, or SSMs), and the other is like analyzing a grid of individual notes (Convolutional Neural Networks, or CNNs).

The Problem:
The authors of this paper noticed that trying to force one type of "eye" to do everything creates blind spots. If you only use the "melody" style, you miss sharp edges. If you only use the "grid" style, you miss the big context. They realized that these two styles are actually complementary—like peanut butter and jelly. They taste better together than apart.

The Solution: "Liquid Fusion" (LFNet)
The team built a new system called LFNet (Liquid Fusion Network). Instead of forcing the two styles to merge into one, they created a special "mixing bowl" inspired by Liquid Neural Networks.

Think of this mixing bowl as a smart bartender:

The "Big Picture" Gazer (VMamba) is the memory of the drink. It holds the continuous flow of information.
The "Detail" Detective (ConvNeXt) is the stimulus or the fresh ingredient being added.
The Liquid Fusion is the gating mechanism. It acts like a smart valve that decides, moment by moment, how much of the "fresh ingredient" to pour in.
- If the scene needs more context, the valve opens to let the "memory" flow through.
- If the scene needs sharp edges, the valve opens to let the "fresh ingredient" (details) mix in.

This happens dynamically, meaning the system changes its mind instantly as it looks at different parts of the image. It's not a static mix; it's a living, breathing blend that adapts to what it sees.

The "Polishing" Step: Saliency-Guided Upsampling
Once the system has mixed the big picture and the details, it needs to zoom back out to show the final result. Usually, when you zoom out or stretch an image in computers, it gets blurry or pixelated (like a low-resolution photo).

The authors added a special tool called Saliency-Guided Upsampling (SGU). Imagine this as a high-tech photo editor that doesn't just stretch the image; it looks at the "frequency" (the rhythm of the image) and the "shape" simultaneously. It ensures that when the image gets bigger, the edges stay sharp and the object doesn't look fuzzy or broken. It keeps the "soul" of the object intact while making it big enough to see.

What Did They Test?
They didn't just test this on simple pictures. They tested it on five different types of "detective missions":

Standard Photos (RGB): Finding objects in normal pictures.
3D Photos (RGB-D): Using depth sensors (like a 3D camera) to find objects.
Thermal Photos (RGB-T): Using heat cameras to find objects in the dark.
Video (VSOD): Finding moving objects in a video stream.
Triple-Mode (VDT): Using visible light, depth, and heat all at once.

The Result:
In every single test, their "Liquid Fusion" system beat the current best methods. It found objects more accurately, drew sharper boundaries, and did it all with a smaller, more efficient brain (fewer parameters) than the heavy, complex systems used before.

In Summary:
The paper claims that by admitting different computer vision styles have different "superpowers" and creating a smart, liquid-like way to mix them on the fly, they built a better, faster, and more accurate way for computers to spot important objects in any kind of scene. They call this the "Liquid Fusion Network."

Technical Summary: Liquid Fusion of Heterogeneous Representations for General Salient Object Detection

1. Problem Statement

General Salient Object Detection (SOD) aims to identify and segment visually prominent objects across diverse scenarios, including single-modal (RGB), dual-modal (RGB-D, RGB-T), video (VSOD), and tri-modal (VDT) inputs. While recent advancements have utilized State Space Models (SSMs) like Mamba to achieve efficient long-range dependency modeling with linear complexity, a critical limitation persists: the neglect of inherent spectral biases across different neural network paradigms.

The authors argue that current approaches often force a single architecture (e.g., a pure SSM) to capture all necessary multi-scale contexts through increasingly complex scanning strategies. However, signal processing analysis reveals that Convolutional Neural Networks (CNNs) and SSMs exhibit distinct and complementary frequency preferences. CNNs tend to capture specific spectral signatures (often high-frequency spatial details), while SSMs excel at continuous sequence-based semantics. Relying solely on one paradigm creates representational blind spots, preventing a model from achieving balanced full-spectrum perception necessary for robust general SOD.

2. Methodology: LFNet

To address these limitations, the authors propose LFNet (Liquid Fusion Network), a heterogeneous hybrid framework that harmonizes representations from SSMs (specifically VMamba) and CNNs (specifically ConvNeXt). The architecture is inspired by the continuous-time dynamics of Liquid Neural Networks (LNNs).

2.1 Heterogeneous Hybrid Encoder

LFNet employs a dual-stream encoder:

VMamba Stream: Extracts continuous state-space representations with linear computational complexity, capturing global sequence-based semantics.
ConvNeXt Stream: Preserves grid-based spatial inductive biases, capturing comprehensive spectral cues without complex scanning engineering.
Modality Fusion Block (MFB): Located within the ConvNeXt stream, this block adapts features based on input modality. It uses a 1×1 projection for RGB, a single Liquid Fusion Module (LFM) for dual-modal tasks (RGB-D/T, VSOD), and a cascaded LFM structure for tri-modal VDT tasks, avoiding the quadratic cost of cross-attention.

2.2 Liquid Fusion Module (LFM)

The core innovation is the LFM, which treats feature integration as a state-stimulus interaction rather than static concatenation or addition.

State: The VMamba feature ( $f^v_i$ ) is treated as an evolving memory state.
Stimulus: The ConvNeXt feature ( $f^c_i$ ) is treated as an exogenous stimulus.
Mechanism: The module employs a dynamic gating mechanism derived from Liquid Time-Constant (LTC) dynamics. It calculates a spatial liquid gate ( $G_i$ $G_{i}$ ) to regulate the injection of the stimulus into the memory state.
- If $G_i \to 1$ , the network prioritizes grid-based spatial cues (ConvNeXt).
- If $G_i \to 0$ , it prioritizes sequence-based context (VMamba).
  This allows for content-aware, dynamic feature aggregation that scales effectively to multi-modal cues.

2.3 Saliency-Guided Upsampling (SGU)

To address spectral aliasing and blurred boundaries common in standard upsampling, LFNet introduces the SGU operator.

Spectral-Spatial Co-Design: The module uses a dual-branch approach. One branch applies a learnable complex weight matrix in the frequency domain (via FFT/IFFT) to preserve global shape integrity. The other branch uses stacked convolutions to capture high-frequency edge cues.
Fusion: The final feature is a fusion of both domains with a residual connection, ensuring sharp boundary recovery during resolution restoration.

2.4 Training Strategy

The model utilizes deep supervision with a weighted combination of Binary Cross Entropy (BCE) and Intersection-over-Union (IoU) losses across all prediction scales.

3. Key Contributions

Spectral Analysis & Hybrid Paradigm: The authors provide a dataset-level spectral analysis demonstrating the inherent complementarity of CNNs and SSMs. They propose LFNet to bridge these spectral biases, moving beyond single-paradigm limitations.
Liquid Fusion Mechanism: They design a novel LFM based on a continuous state-stimulus paradigm. This dynamic gating mechanism effectively aggregates heterogeneous features and serves as a scalable solution for multi-modal fusion (RGB, RGB-D, RGB-T, VSOD, VDT).
Saliency-Guided Upsampling: They develop the SGU operator, which utilizes a dual-branch spectral-spatial design to propagate semantics and suppress artifacts during upsampling.
State-of-the-Art Performance: Extensive experiments across five diverse tasks demonstrate that LFNet achieves superior performance compared to CNN-based, Transformer-based, and Mamba-based methods, often with a more favorable trade-off between accuracy and model efficiency.

4. Experimental Results

The paper evaluates LFNet on five general SOD tasks using standard benchmarks:

RGB SOD: Tested on DUTS, DUT-O, HKU-IS, PASCAL-S, and ECSSD. LFNet outperforms heavy Transformer models (e.g., ICON-S) and recent Mamba-based methods (Samba) while using significantly fewer parameters (43.23M vs. 94.30M for ICON-S).
RGB-D SOD: Tested on NJUD, NLPR, SIP, STERE, and DUTLF-D. LFNet surpasses competitors like VST-S++ with only ~31% of the parameters.
RGB-T SOD: Tested on VT821, VT1000, and VT5000. LFNet achieves new SOTA results, outperforming Samba across all benchmarks.
VSOD: Tested on DAVIS, DAVSOD-easy, FBMS, SegV2, and VOS. LFNet shows robustness in dynamic scenes, outperforming VSCode-S.
VDT SOD: Tested on VDT-2048. LFNet establishes a new SOTA with an $S_m$ of 94.2%.

Ablation Studies confirm that:

The heterogeneous hybrid design outperforms single-stream (VMamba-only or ConvNeXt-only) and naive dual-stream baselines.
The Liquid Fusion Module is more effective than additive fusion, concatenation, or cross-attention.
The SGU operator yields consistent gains over standard bilinear interpolation and transposed convolutions.

5. Significance and Claims

The paper claims that the fundamental nature of SOD demands robust representations across the entire frequency spectrum. By harmonizing the inherent spectral biases of CNNs and SSMs through a liquid fusion paradigm, LFNet offers a flexible and efficient solution for general SOD.

The authors emphasize that their work represents the first attempt to leverage liquid dynamics (specifically the state-stimulus paradigm of LNNs) to construct a cross-paradigm heterogeneous fusion framework for dense visual prediction. The significance lies in demonstrating that a hybrid architecture, dynamically balancing global semantics and local details, can outperform homogeneous architectures (whether pure CNN, Transformer, or SSM) across single, dual, and tri-modal tasks without incurring quadratic computational costs. The code is released to facilitate further research in this direction.