⚡ electrical engineering

Axial-Relation Guided Fusion State Space Model for Optical-Elevation Sensing Image Segmentation

The paper proposes ARG-Mamba, a state space model-based framework that utilizes multi-scale context modeling and axial-relation guided fusion to achieve superior performance and computational efficiency in optical-elevation remote sensing image segmentation.

Original authors: Feng Gao, Zhilin Jin, Yanhai Gan, Junyu Dong, Qian Du

Published 2026-05-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Feng Gao, Zhilin Jin, Yanhai Gan, Junyu Dong, Qian Du

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, high-resolution jigsaw puzzle of a city, but you have two different sets of clues to help you.

The Two Sets of Clues:

The Optical Photo: This is like a standard, colorful photograph taken from a plane. It shows you the colors, textures, and patterns of the ground (like green grass, gray roads, or red roofs).
The Elevation Map (DSM): This is like a 3D topographic map. It doesn't show colors, but it tells you exactly how high or low things are. It knows a tree is tall, a car is short, and a building is very tall.

The Problem:
For a long time, computer programs trying to sort these images (telling the difference between a "tree" and a "car") mostly relied on just the colorful photo. They often got confused. For example, a flat gray roof might look like a gray road in a photo. Or a small car might get lost in the texture of a parking lot.

The authors of this paper realized that if you combine the "color clues" with the "height clues," the computer should get much better at solving the puzzle. However, existing methods were bad at two things:

They couldn't see the "big picture" (the whole city layout) and the "tiny details" (a single car) at the same time.
They weren't very good at mixing the color and height information together in a smart way.

The Solution: ARG-Mamba
The authors built a new AI system called ARG-Mamba. Think of it as a super-smart detective that uses a special toolkit to solve the puzzle.

1. The "Multi-Scale" Detective (MS-SSM)
Imagine you are looking at a map. Sometimes you need to zoom out to see the whole neighborhood, and sometimes you need to zoom in to see a single mailbox.

Old methods usually looked at the map at just one zoom level.
ARG-Mamba uses a "Multi-Scale State Space Module." This is like a detective who can instantly switch between a wide-angle lens and a magnifying glass. It looks at the image at four different sizes at once. This allows it to understand that a "car" is a small object sitting on a "road" that is part of a larger "city." It does this very quickly, without getting bogged down in slow calculations.

2. The "Axial-Relation" Mixer (ARGFM)
Now, how do you mix the color photo with the height map?

Old methods often just mashed the two images together, like throwing two bowls of soup into one pot and hoping they mix well.
ARG-Mamba uses an "Axial-Relation Guided Fusion Module." Imagine the image is a giant grid of squares (like a chessboard). This module looks at the grid in two specific directions: Rows (horizontal) and Columns (vertical).
- It asks: "If I look across this row, does the color match the height?"
- It asks: "If I look down this column, do these two clues belong together?"
- By breaking the complex mixing job into these simple horizontal and vertical lines, the computer can efficiently figure out exactly how the color and height data relate to each other. It's like organizing a messy room by first lining up all the books on the shelves (rows) and then stacking the boxes (columns), rather than trying to tidy everything at once.

The Results
The authors tested this new detective on two famous datasets (Vaihingen and Potsdam), which are like standardized "exams" for remote sensing AI.

Accuracy: ARG-Mamba got higher scores than all the other top competitors. It was particularly good at spotting small, tricky things like cars and distinguishing between trees and low bushes.
Speed: Despite being smarter, it was also faster and required less computer power than many of the other heavy-duty methods.

In Summary
This paper presents a new way for computers to look at aerial photos and 3D height maps together. By using a system that can zoom in and out simultaneously and by mixing the data in a structured, row-by-row and column-by-column fashion, the new AI (ARG-Mamba) creates a much clearer and more accurate map of the world than previous methods.

Technical Summary: Axial-Relation Guided Fusion State Space Model for Optical-Elevation Sensing Image Segmentation

Problem Statement

Semantic segmentation of high-resolution remote sensing imagery is critical for Earth observation tasks such as urban planning and disaster management. However, relying solely on optical imagery often yields suboptimal results in complex scenes due to limited spectral and textural information. While Digital Surface Model (DSM) elevation data provides complementary geometric and structural cues, existing multi-modal fusion methods face two primary challenges:

Insufficient Multi-Scale Modeling: Most State Space Model (SSM) based methods operate on single-scale token sequences, struggling to simultaneously capture fine-grained local details and coarse global contextual dependencies inherent in remote sensing objects.
Suboptimal Cross-Modal Fusion: The semantic relevance between optical (spectral/texture) and elevation (geometric/structural) features varies across object categories. Existing fusion strategies often fail to adaptively model these complex, context-aware cross-modal correlations.

Methodology: ARG-Mamba

The authors propose ARG-Mamba (Axial-Relation Guided Fusion Mamba), a dual-stream encoder-decoder framework based on State Space Models (SSMs) designed for optical-elevation semantic segmentation. The architecture integrates linear-complexity long-range dependency modeling with explicit cross-modal fusion.

1. Multi-Scale State Space Module (MS-SSM)

To address scale variation, the authors introduce the MS-SSM within the encoder's Visual State Space Blocks (VSSBs).

Mechanism: The module splits input feature maps into multiple sub-features along the channel dimension, each processed at different spatial scales (e.g., $\{1, 2, 4, 8\}$ ).
Scanning: It employs a Multi-Scale Selective Scan (MS-SS2D) that performs two-way scanning within non-overlapping windows for each scale. This restricts scanning distance to minimize noise while capturing local structures.
Fusion: Features from all scales are concatenated and processed via linear projection and depth-wise convolution (DConv) to adaptively merge multi-scale cues and reconcile channel redundancies. This allows the model to capture both local details and global dependencies with linear computational complexity.

2. Axial-Relation Guided Fusion Module (ARGFM)

Inserted between corresponding stages of the optical and elevation encoders, the ARGFM facilitates structured cross-modal interaction.

Axial Relation Attention Layer (ARAL): Instead of computing a full 2D attention matrix (which has $O((HW)^2)$ complexity), ARAL decomposes the dense relation matrix between optical and elevation tokens into row-wise and column-wise axial attentions. This reduces complexity to $O(HW(H+W))$ while capturing interaction patterns aligned with grid-like remote sensing structures (e.g., roads, buildings).
Fusion SSM (Fus-SSM): The relation-aware tokens generated by ARAL act as modulators for a fusion SSM block. This block cross-concatenates local context tokens and relation tokens, using selective scanning to propagate information and facilitate deep interaction between spectral and geometric cues.
Integration: The refined features are passed through cross-attention and linear projections to produce fused representations for the decoder, while modality-specific features continue to the next encoder stage.

Key Contributions

The paper outlines three primary contributions:

ARG-Mamba Framework: A novel framework integrating SSMs with explicit axial-relation guided fusion to efficiently capture long-range dependencies in multi-source remote sensing images.
MS-SSM: A module designed to handle scale variations in high-resolution imagery, enabling the joint modeling of fine-grained local details and global context within a unified, linear-complexity framework.
ARGFM: A mechanism that explicitly models global cross-modal correlations via axial decomposition, enabling efficient and structured fusion of heterogeneous optical and elevation features.

Experimental Results

The authors evaluated ARG-Mamba on two benchmark datasets: ISPRS Vaihingen (9 cm GSD) and ISPRS Potsdam (5 cm GSD), both containing six land-cover classes.

Quantitative Performance: ARG-Mamba consistently outperformed six state-of-the-art methods (including PSTNet, FEANet, GMNet, CMNeXt, FTransUNet, and MFNet) across all metrics (F1-score, Overall Accuracy, and mIoU).
- On Vaihingen, it achieved an mIoU of 83.72% (vs. 82.42% for the next best, MFNet).
- On Potsdam, it achieved an mIoU of 85.30% (vs. 85.10% for MFNet).
- Notable improvements were observed in detecting small and complex objects like "Cars" and "Low Vegetation."
Qualitative Analysis: Visual results showed ARG-Mamba produced segmentation maps with clearer object boundaries and more consistent region labeling, particularly in building extraction and small object recognition, with reduced clutter noise compared to CNN and Transformer baselines.
Computational Efficiency: The model demonstrated favorable efficiency, requiring significantly fewer parameters (77.1M) and FLOPs (56.4G) than heavy hybrid models like MFNet (313.2M params) and FEANet (255.3M params), while maintaining faster inference times (0.268s).

Significance and Claims

The paper claims that ARG-Mamba offers a balanced solution for optical-elevation segmentation by addressing the limitations of existing multi-scale and multi-modal approaches. By leveraging the linear complexity of SSMs and the structural efficiency of axial-relation decomposition, the framework achieves state-of-the-art accuracy without the prohibitive computational costs associated with Transformer-based or heavy hybrid architectures. The authors assert that their approach is particularly well-suited for large-scale remote sensing tasks where both high precision and computational efficiency are required. The code is made publicly available to support further research in this domain.