让图像匹配提速54%，实现毫秒级实时3D重建。
Speedy MASt3R
- 用FlashAttention+张量融合优化计算图，加速匹配推理。
- 将每对图像延迟从198毫秒降至91毫秒，精度不变。
- 适合需要实时3D感知的MR导航与大场景重建应用。
图像匹配是现代3D视觉算法的核心，对精确场景重建和定位至关重要。MASt3R通过引入快速互匹配机制，结合DUSt3R将匹配任务重构为3D问题，显著提升效率并保持理论保证，已获广泛引用。然而，其在A40 GPU上的单对图像延迟仍高达198.16毫秒，主要源于ViT编码器-解码器与Fast Reciprocal Nearest Neighbor（FastNN）匹配的计算开销。为此，本文提出Speedy MASt3R——一种后训练优化框架，集成FlashMatch（基于FlashAttention v2与分块策略）、通过TensorRT自动调优的层/张量融合（GraphFusion）、内存访问从二次方降至线性的FastNN-Lite优化流水线，以及支持FP16/FP32混合计算的HybridCast。在Aachen Day-Night、InLoc、7-Scenes、ScanNet1500与MegaDepth1500数据集上评估，推理时间减少54%（198毫秒→91毫秒），精度无损。该进展推动了实时3D理解，适用于混合现实导航与大规模3D场景重建。
原文摘要 · Abstract (English)
Image matching is a key component of modern 3D vision algorithms, essential for accurate scene reconstruction and localization. MASt3R redefines image matching as a 3D task by leveraging DUSt3R and introducing a fast reciprocal matching scheme that accelerates matching by orders of magnitude while preserving theoretical guarantees. This approach has gained strong traction, with DUSt3R and MASt3R collectively cited over 250 times in a short span, underscoring their impact. However, despite its accuracy, MASt3R's inference speed remains a bottleneck. On an A40 GPU, latency per image pair is 198.16 ms, mainly due to computational overhead from the ViT encoder-decoder and Fast Reciprocal Nearest Neighbor (FastNN) matching. To address this, we introduce Speedy MASt3R, a post-training optimization framework that enhances inference efficiency while maintaining accuracy. It integrates multiple optimization techniques, including FlashMatch-an approach leveraging FlashAttention v2 with tiling strategies for improved efficiency, computation graph optimization via layer and tensor fusion having kernel auto-tuning with TensorRT (GraphFusion), and a streamlined FastNN pipeline that reduces memory access time from quadratic to linear while accelerating block-wise correlation scoring through vectorized computation (FastNN-Lite). Additionally, it employs mixed-precision inference with FP16/FP32 hybrid computations (HybridCast), achieving speedup while preserving numerical precision. Evaluated on Aachen Day-Night, InLoc, 7-Scenes, ScanNet1500, and MegaDepth1500, Speedy MASt3R achieves a 54% reduction in inference time (198 ms to 91 ms per image pair) without sacrificing accuracy. This advancement enables real-time 3D understanding, benefiting applications like mixed reality navigation and large-scale 3D scene reconstruction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。