为3D点云模型设计了首个适配Mamba架构的高效微调方法
Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models

- 通过状态感知适配器实现对Mamba状态更新的轻量级控制
- 仅用5%可训练参数即达到与全量微调相当的性能
- 适合需要高效部署3D点云模型的研究者和工程师
预训练的3D点云基础模型(PFMs)在多种下游任务中展现出强大的迁移能力。然而,对这些模型进行全量微调计算开销大且存储需求高。参数高效微调(PEFT)提供了可行替代方案,但现有方法主要针对Transformer骨干网络,依赖于标记级提示或特征变换。Mamba骨干网络在标记级适应与状态级序列动态之间存在粒度不匹配,直接将现有PEFT方法应用于冻结的Mamba骨干会导致显著性能下降和优化不稳定。为此,我们提出Mantis,首个专为3D PFMs设计的Mamba原生PEFT框架。具体而言,引入状态感知适配器(SAA),在选择性状态空间更新中注入轻量级任务相关控制信号,实现状态级适应,同时保持预训练主干冻结。此外,通过双序列化一致性蒸馏(DSCD)对不同有效的点云序列化方式进行正则化,降低序列化带来的不稳定性。在多个基准上的大量实验表明,我们的Mantis仅需约5%的可训练参数即可实现竞争性性能。代码已开源:https://github.com/gzhhhhhhh/Mantis。
原文摘要 · Abstract (English)
Pre-trained 3D point cloud foundation models (PFMs) have demonstrated strong transferability across diverse downstream tasks. However, full fine-tuning these models is computationally expensive and storage-intensive. Parameter-efficient fine-tuning (PEFT) offers a promising alternative, but existing PEFT approaches are primarily designed for Transformer-based backbones and rely on token-level prompting or feature transformation. Mamba-based backbones introduce a granularity mismatch between token-level adaptation and state-level sequence dynamics. Consequently, straightforward transfer of existing PEFT approaches to frozen Mamba backbones leads to substantial accuracy degradation and unstable optimization. To address this issue, we propose Mantis, the first Mamba-native PEFT framework for 3D PFMs. Specifically, a State-Aware Adapter (SAA) is introduced to inject lightweight task-conditioned control signals into selective state-space updates, enabling state-level adaptation while keeping the pre-trained backbone frozen. Moreover, different valid point cloud serializations are regularized by Dual-Serialization Consistency Distillation (DSCD), thereby reducing serialization-induced instability. Extensive experiments across multiple benchmarks demonstrate that our Mantis achieves competitive performance with only about 5% trainable parameters. Our code is available at https://github.com/gzhhhhhhh/Mantis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。