用Mamba结构实现轻量高效特征匹配，性能超越主流方法。
JamMa: Ultra-lightweight Local Feature Matching with Joint Mamba
- 采用联合扫描与跳步策略，实现图像间高频交互和高效计算。
- 参数与浮点运算量不足50%，匹配精度仍超注意力机制模型。
- 单卡训练、低延迟推理，适合移动端或实时系统部署。
现有顶尖特征匹配器依赖Transformer捕捉长程依赖，但面临高空间复杂度问题，导致训练耗时、推理延迟高。为平衡性能与效率，我们提出基于Mamba的超轻量匹配器JamMa，其具有线性复杂度O(N)。通过引入联合扫描-合并策略JEGO，实现：（1）双图联合扫描以增强高频互信息；（2）跳步扫描降低序列长度；（3）全局感受野；（4）全向特征表示。JEGO在特征匹配任务中显著优于VMamba与EVMamba的扫描-合并策略。相比基于注意力的稀疏与半密集匹配器，JamMa以少于50%的参数量与浮点运算量，实现了更优的性能-效率平衡。
原文摘要 · Abstract (English)
Existing state-of-the-art feature matchers capture long-range dependencies with Transformers but are hindered by high spatial complexity, leading to demanding training and highlatency inference. Striking a better balance between performance and efficiency remains a challenge in feature matching. Inspired by the linear complexity O(N) of Mamba, we propose an ultra-lightweight Mamba-based matcher, named JamMa, which converges on a single GPU and achieves an impressive performance-efficiency balance in inference. To unlock the potential of Mamba for feature matching, we propose Joint Mamba with a scan-merge strategy named JEGO, which enables: (1) Joint scan of two images to achieve high-frequency mutual interaction, (2) Efficient scan with skip steps to reduce sequence length, (3) Global receptive field, and (4) Omnidirectional feature representation. With the above properties, the JEGO strategy significantly outperforms the scan-merge strategies proposed in VMamba and EVMamba in the feature matching task. Compared to attention-based sparse and semi-dense matchers, JamMa demonstrates a superior balance between performance and efficiency, delivering better performance with less than 50% of the parameters and FLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。