用混合Mamba与Transformer实现高效半稠密特征匹配
VMatcher: State-Space Semi-Dense Local Feature Matching
- 融合Mamba长序列处理与Transformer注意力机制
- 线性复杂度下达到新基准，推理速度显著提升
- 适合实时应用，兼顾效率与匹配鲁棒性
本文提出VMatcher，一种基于Mamba-Transformer的混合网络，用于图像对之间的半稠密特征匹配。现有基于学习的匹配方法虽性能领先，但依赖计算开销大的Transformer注意力机制（二次复杂度）。相比之下，Mamba引入选择性状态空间模型（SSM），以线性复杂度实现相当或更优性能，大幅提升效率。VMatcher结合Mamba的高效长序列处理能力与Transformer的注意力机制，设计多种配置（如分层结构），在保持高鲁棒性和实用性的同时，显著提升推理速度，为实时应用设定新基准。源代码已公开于https://github.com/ayoussf/VMatcher。
原文摘要 · Abstract (English)
This paper introduces VMatcher, a hybrid Mamba-Transformer network for semi-dense feature matching between image pairs. Learning-based feature matching methods, whether detector-based or detector-free, achieve state-of-the-art performance but depend heavily on the Transformer's attention mechanism, which, while effective, incurs high computational costs due to its quadratic complexity. In contrast, Mamba introduces a Selective State-Space Model (SSM) that achieves comparable or superior performance with linear complexity, offering significant efficiency gains. VMatcher leverages a hybrid approach, integrating Mamba's highly efficient long-sequence processing with the Transformer's attention mechanism. Multiple VMatcher configurations are proposed, including hierarchical architectures, demonstrating their effectiveness in setting new benchmarks efficiently while ensuring robustness and practicality for real-time applications where rapid inference is crucial. Source Code is available at: https://github.com/ayoussf/VMatcher
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。