用状态空间模型实现高效多视角立体重建，速度与精度双提升。
MVSMamba: Multi-View Stereo with State Space Model
- 引入Mamba架构替代Transformer，线性复杂度实现全局特征聚合。
- 在DTU和Tanks-and-Temples上优于当前最优方法，性能与效率俱佳。
- 设计动态扫描模块，支持跨视角、多尺度的特征交互与建模。
基于学习的多视角立体（MVS）依赖于鲁棒的特征表示，关键在于精确的特征匹配。现有MVS方法利用Transformer捕捉局部特征提取器生成的特征间的长程依赖，但其二次复杂度导致性能与效率难以平衡。受Mamba架构全局建模能力与线性复杂度启发，我们提出首个基于Mamba的MVS网络——MVSMamba，实现高效全局特征聚合且计算开销极小。为充分挖掘Mamba在MVS中的潜力，我们设计了基于新颖参考中心动态扫描策略的动态Mamba模块（DM-module），可实现：(1) 从参考视图到源视图的高效视间与视内特征交互；(2) 全向多视角特征表示；(3) 多尺度全局特征聚合。大量实验表明，MVSMamba在DTU数据集和Tanks-and-Temples基准上均超越当前最优方法，在性能与效率方面表现更优。代码已开源。
原文摘要 · Abstract (English)
Robust feature representations are essential for learning-based Multi-View Stereo (MVS), which relies on accurate feature matching. Recent MVS methods leverage Transformers to capture long-range dependencies based on local features extracted by conventional feature pyramid networks. However, the quadratic complexity of Transformer-based MVS methods poses challenges to balance performance and efficiency. Motivated by the global modeling capability and linear complexity of the Mamba architecture, we propose MVSMamba, the first Mamba-based MVS network. MVSMamba enables efficient global feature aggregation with minimal computational overhead. To fully exploit Mamba's potential in MVS, we propose a Dynamic Mamba module (DM-module) based on a novel reference-centered dynamic scanning strategy, which enables: (1) Efficient intra- and inter-view feature interaction from the reference to source views, (2) Omnidirectional multi-view feature representations, and (3) Multi-scale global feature aggregation. Extensive experimental results demonstrate MVSMamba outperforms state-of-the-art MVS methods on the DTU dataset and the Tanks-and-Temples benchmark with both superior performance and efficiency. The source code is available at https://github.com/JianfeiJ/MVSMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。