Technical Summary: Mantis - Mamba-native Tuning for 3D Point Cloud Foundation Models

1. Problem Statement

Pre-trained 3D Point Cloud Foundation Models (PFMs) have demonstrated strong transferability across downstream tasks. However, full fine-tuning of these models is computationally expensive and storage-intensive. While Parameter-Efficient Fine-Tuning (PEFT) methods offer a solution, existing approaches (e.g., Instance-aware Dynamic Prompt Tuning, Adapters) are primarily designed for Transformer-based backbones.

The authors identify a fundamental granularity mismatch when applying these Transformer-native PEFT methods to Mamba-based backbones (State Space Models):

1. Token-level vs. State-level: Transformer PEFT operates on token-level prompts or feature transformations. In contrast, Mamba relies on state-level sequence dynamics and recurrent hidden-state propagation. Directly transferring token-level adaptation to frozen Mamba backbones leads to substantial accuracy degradation and unstable optimization.
2. Serialization Sensitivity: Mamba models process point clouds as sequential inputs. The hidden-state propagation is inherently sensitive to the specific traversal order (serialization) of the points. Existing PEFT modules often lack mechanisms to handle this order-sensitivity, leading to inconsistent representations and unstable predictions when the backbone is frozen.

2. Methodology: Mantis

The paper proposes Mantis, the first Mamba-native PEFT framework designed specifically for 3D point cloud analysis. It consists of two complementary components:

A. State-Aware Adapter (SAA)

To address the granularity mismatch, SAA injects lightweight, task-conditioned control signals directly into the state-space updates of the Mamba backbone, rather than modifying tokens or features.

• Mechanism: SAA modulates the state transition operators ($A_t, B_t, C_t, \Delta_t$) of the selective SSM. It takes the current token, the previous hidden state, and an order-aware representation as inputs.
• Control Signal Generation: The control signal is generated via a proximal optimization problem, yielding a sparse, soft-thresholded signal. This ensures the adaptation is low-rank and stable.
• Operator Modulation: The sparse control signal is used to perturb the continuous-time operators via low-rank matrices ($U, V$), which are then discretized (via Zero-Order Hold) to update the discrete-time system. This allows the frozen backbone to adapt its state evolution dynamically without altering the pre-trained weights directly.

B. Dual-Serialization Consistency Distillation (DSCD)

To address the serialization sensitivity of Mamba, DSCD enforces consistency across two valid serializations of the same point cloud.

• Dual-Order Pipeline: The input point cloud is serialized along two complementary space-filling curves (e.g., Hilbert curve and its transposed variant) to produce two distinct sequences.
• Consistency Loss: During training, DSCD aligns the feature representations and predictive distributions of these two branches.
  • Feature Consistency ($L_{feat}$): Minimizes the distance between normalized global embeddings of the two branches.
  • Prediction Consistency ($L_{pred}$): Minimizes the KL-divergence between the softened predictive distributions of the two branches.
• Objective: This regularization stabilizes downstream optimization by ensuring the model learns representations that are robust to the specific traversal order, preserving pre-trained geometric priors.

3. Key Contributions

1. Identification of the Gap: The authors reveal the fundamental incompatibility between Transformer-native PEFT methods and Mamba-based backbones, attributing performance drops to granularity mismatches and serialization sensitivity.
2. Mantis Framework: The proposal of the first Mamba-native PEFT framework, featuring:
   • SAA: A state-level adapter that modulates selective SSM operators via sparse, input-dependent control signals.
   • DSCD: A consistency distillation mechanism that regularizes cross-serialization instability.
3. Theoretical Analysis: The paper provides theoretical bounds on the hidden-state deviation induced by SAA, demonstrating that the adaptation remains controlled and stable through sparse low-rank perturbations.
4. Empirical Validation: Extensive experiments showing Mantis outperforms full fine-tuning and existing PEFT methods while using significantly fewer parameters.

4. Experimental Results

The authors evaluated Mantis on three pre-trained Mamba backbones (PointMamba, Mamba3D, ZigzagPointMamba) across multiple benchmarks:

• Object Classification (ScanObjectNN & ModelNet40):
  • On the challenging ScanObjectNN PB_T50_RS split, Mantis (with only ~5% trainable parameters) surpassed full fine-tuning baselines. For instance, on Mamba3D, Mantis achieved 93.48% accuracy compared to 92.05% for full fine-tuning, using only 0.8M parameters versus 16.9M.
  • Mantis consistently outperformed direct transfers of existing PEFT methods (like IDPT, PMA, LoRA), which suffered significant performance degradation on Mamba backbones.
• Part Segmentation (ShapeNetPart):
  • Mantis achieved competitive instance-level mIoU and notable improvements in class-level mIoU compared to full fine-tuning, demonstrating effectiveness in fine-grained geometric structure modeling.
• Few-Shot Learning:
  • In few-shot experiments on ModelNet40, Mantis achieved the best or second-best accuracy across various $N$-way $M$-shot configurations.
• Efficiency:
  • Mantis maintains the linear-complexity modeling paradigm of Mamba. It achieves a superior trade-off between trainable parameters and FLOPs compared to full fine-tuning and other PEFT approaches.
  • Training stability analysis shows Mantis converges smoothly, whereas direct transfers of token-level prompts (e.g., IDPT) exhibit unstable training dynamics.

5. Significance and Claims

The paper claims that Mantis is a necessary evolution for adapting 3D foundation models as the field shifts from Transformers to State Space Models (Mamba).

• Necessity of Native Design: The authors argue that simply reusing Mamba modules as external adapters (as done in some prior works like PMA) or applying Transformer-style PEFT to Mamba backbones is insufficient. True efficiency requires adaptation mechanisms that align with the state-level dynamics of Mamba.
• Robustness: The framework is particularly effective in noisy, real-world scenarios (e.g., ScanObjectNN PB_T50_RS) where serialization-induced instability is most pronounced.
• Future Baseline: The authors position Mantis as a baseline and a source of insight for future research into Mamba-native PEFT, highlighting that state-level adaptation is critical for unlocking the potential of frozen Mamba backbones.

The work concludes that by addressing the specific architectural constraints of Mamba (state dynamics and serialization sensitivity), Mantis enables efficient, high-performance adaptation of 3D PFMs with minimal computational overhead.