在特征匹配中，让描述符先学习旋转不变性，比后期再处理更高效且效果相当。
Who Handles Orientation? Investigating Invariance in Feature Matching
- 让描述符阶段就学习旋转不变性，可提前实现鲁棒匹配
- 训练规模越大，对旋转图像的泛化能力越强，提升显著
- 新模型在多模态、极端和卫星图像匹配上达到顶尖水平
在3D计算机视觉中，图像间关键点匹配是核心问题，但现代匹配器对大范围平面内旋转敏感。一种常见方法是通过数据增强学习旋转不变性，但其应在哪一阶段引入仍不明确。本文在现代稀疏匹配流程中系统研究此问题，基于大规模3D视觉数据集训练，并在多个主流图像匹配基准上评估。结果表明，将旋转不变性提前至描述符阶段，性能与后期在匹配器中处理相当；但前者使匹配过程更早具备旋转鲁棒性，从而加速整体匹配。此外，大规模训练下引入旋转不变性不会损害正立图像性能。我们还发现，随着训练数据量增加，旋转不变性显著增强。本文发布两个对平面内旋转鲁棒的匹配器，在多模态（WxBS）、极端（HardMatch）及卫星图像匹配（SatAst）任务上均达当前最优。代码已开源：https://github.com/davnords/loma。
原文摘要 · Abstract (English)
Finding matching keypoints between images is a core problem in 3D computer vision. However, modern matchers struggle with large in-plane rotations. A straightforward mitigation is to learn rotation invariance via data augmentation. However, it remains unclear at which stage rotation invariance should be incorporated. In this paper, we study this in the context of a modern sparse matching pipeline. We perform extensive experiments by training on a large collection of 3D vision datasets and evaluating on popular image matching benchmarks. Surprisingly, we find that incorporating rotation invariance already in the descriptor yields similar performance to handling it in the matcher. However, rotation invariance is achieved earlier in the matcher when it is learned in the descriptor, allowing for a faster rotation-invariant matcher. Further, we find that enforcing rotation invariance does not hurt upright performance when trained at scale. Finally, we study the emergence of rotation invariance through scale and find that increasing the training data size substantially improves generalization to rotated images. We release two matchers robust to in-plane rotations that achieve state-of-the-art performance on e.g. multi-modal (WxBS), extreme (HardMatch), and satellite image matching (SatAst). Code is available at https://github.com/davnords/loma.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。