arXiv:2511.21265cs.CV2025-11

用3D高斯点云生成精准匹配数据,实现零样本图像匹配新突破

Unlocking Zero-shot Potential of Semi-dense Image Matching via Gaussian Splatting

  • 通过几何校正生成高精度对应关系数据
  • 使匹配器在极端视角下仍保持17.7%性能提升
  • 适合需要零样本匹配的视觉系统研究者

基于学习的图像匹配严重依赖大规模、多样且几何精确的训练数据。3D高斯点云(3DGS)可实现逼真的新视角合成,因而成为数据生成的理想选择。然而其几何不准确和深度渲染偏差限制了可靠对应标注。为此,我们提出MatchGS,首个系统性修正并利用3DGS实现鲁棒零样本图像匹配的框架。方法包含两部分:(1) 几何忠实的数据生成流程,精修3DGS几何结构以生成高精度对应标签,支持大规模多样化视角合成而不损失渲染保真度;(2) 2D-3D表示对齐策略,将3DGS的显式3D知识注入2D匹配器,引导其学习视角不变的3D表示。生成的真实标签使视差误差降低达40倍,支持极端视角监督,并通过高斯属性提供自监督信号。仅使用本数据训练的前沿匹配器在公开基准上实现显著零样本性能提升,最高达17.7%。结果表明,经适当几何修正后,3DGS可作为可扩展、高保真、结构丰富的数据源,开启新一代鲁棒零样本图像匹配的新范式。

原文摘要 · Abstract (English)

Learning-based image matching critically depends on large-scale, diverse, and geometrically accurate training data. 3D Gaussian Splatting (3DGS) enables photorealistic novel-view synthesis and thus is attractive for data generation. However, its geometric inaccuracies and biased depth rendering currently prevent robust correspondence labeling. To address this, we introduce MatchGS, the first framework designed to systematically correct and leverage 3DGS for robust, zero-shot image matching. Our approach is twofold: (1) a geometrically-faithful data generation pipeline that refines 3DGS geometry to produce highly precise correspondence labels, enabling the synthesis of a vast and diverse range of viewpoints without compromising rendering fidelity; and (2) a 2D-3D representation alignment strategy that infuses 3DGS' explicit 3D knowledge into the 2D matcher, guiding 2D semi-dense matchers to learn viewpoint-invariant 3D representations. Our generated ground-truth correspondences reduce the epipolar error by up to 40 times compared to existing datasets, enable supervision under extreme viewpoint changes, and provide self-supervisory signals through Gaussian attributes. Consequently, state-of-the-art matchers trained solely on our data achieve significant zero-shot performance gains on public benchmarks, with improvements of up to 17.7%. Our work demonstrates that with proper geometric refinement, 3DGS can serve as a scalable, high-fidelity, and structurally-rich data source, paving the way for a new generation of robust zero-shot image matchers.

图像匹配3D高斯零样本数据生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。