将单张2D图像升维到3D空间，实现更鲁棒的特征匹配。
Learning Dense Feature Matching via Lifting Single 2D Image to 3D Space
- 通过合成多视角图像和3D特征高斯表示训练3D感知编码器。
- 在大量单视图合成数据上训练解码器，实现跨域零样本泛化。
- 适合需要高鲁棒性特征匹配的复杂场景应用。
特征匹配在众多计算机视觉任务中扮演基础角色，但现有方法严重依赖稀少且高质量的多视角图像集，限制了其在多样化、挑战性场景中的泛化能力。传统特征编码器通常在单视图2D图像上训练，难以捕捉3D一致性对应关系。本文提出一种两阶段框架——Lift to Match (L2M)，将单张2D图像升维至3D空间，充分利用大规模且多样的单视图图像数据。第一阶段，结合多视角图像合成与3D特征高斯表示，训练一个具备3D几何先验的特征编码器；第二阶段，采用新视角渲染策略，并基于单视图图像生成大规模合成数据，训练特征解码器以实现鲁棒特征匹配，从而在跨域场景中获得良好泛化能力。大量实验表明，该方法在零样本评估基准上表现优异，验证了框架的有效性。
原文摘要 · Abstract (English)
Feature matching plays a fundamental role in many computer vision tasks, yet existing methods heavily rely on scarce and clean multi-view image collections, which constrains their generalization to diverse and challenging scenarios. Moreover, conventional feature encoders are typically trained on single-view 2D images, limiting their capacity to capture 3D-aware correspondences. In this paper, we propose a novel two-stage framework that lifts 2D images to 3D space, named as \textbf{Lift to Match (L2M)}, taking full advantage of large-scale and diverse single-view images. To be specific, in the first stage, we learn a 3D-aware feature encoder using a combination of multi-view image synthesis and 3D feature Gaussian representation, which injects 3D geometry knowledge into the encoder. In the second stage, a novel-view rendering strategy, combined with large-scale synthetic data generation from single-view images, is employed to learn a feature decoder for robust feature matching, thus achieving generalization across diverse domains. Extensive experiments demonstrate that our method achieves superior generalization across zero-shot evaluation benchmarks, highlighting the effectiveness of the proposed framework for robust feature matching.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。