用3D高斯点云生成中间视角,解决航拍与地面图像匹配难题。
Aerial-Ground Image Feature Matching via 3D Gaussian Splatting-based Intermediate View Rendering
- 通过航拍图重建稀疏模型,用3D高斯点云渲染中间视角。
- 相比传统方法,初始匹配数提升明显,精修后匹配更可靠。
- 适合需要融合航拍与地面数据的三维建模场景。
航拍与地面图像融合在复杂场景三维建模中具有潜力,但受限于可靠对应点的获取。本文提出一种针对航拍与地面图像的特征匹配算法,核心思路是生成中间视角以缓解视角剧烈变化带来的透视畸变。首先,仅利用航拍图像,通过增量式结构光(SfM)引擎重建稀疏模型;其次,采用3D高斯点云(3D Gaussian Splatting)对场景进行渲染,输入为稀疏点和定向图像;设计了一种基于航拍相机姿态的渲染视角确定算法,生成高质量中间图像,弥合航拍与地面图像之间的差距;第三,借助中间图像,在渲染-航拍与渲染-地面图像对间进行可靠特征匹配,并通过中间视图传递对应关系,生成最终匹配。在真实航拍与地面数据集上验证了该方法在特征匹配与场景渲染方面的有效性,实验结果表明:所提方案显著增加了初始与精修匹配数量,能提供足够匹配实现精确ISfM重建及完整的3DGS场景渲染。
原文摘要 · Abstract (English)
The integration of aerial and ground images has been a promising solution in 3D modeling of complex scenes, which is seriously restricted by finding reliable correspondences. The primary contribution of this study is a feature matching algorithm for aerial and ground images, whose core idea is to generate intermediate views to alleviate perspective distortions caused by the extensive viewpoint changes. First, by using aerial images only, sparse models are reconstructed through an incremental SfM (Structure from Motion) engine due to their large scene coverage. Second, 3D Gaussian Splatting is then adopted for scene rendering by taking as inputs sparse points and oriented images. For accurate view rendering, a render viewpoint determination algorithm is designed by using the oriented camera poses of aerial images, which is used to generate high-quality intermediate images that can bridge the gap between aerial and ground images. Third, with the aid of intermediate images, reliable feature matching is conducted for match pairs from render-aerial and render-ground images, and final matches can be generated by transmitting correspondences through intermediate views. By using real aerial and ground datasets, the validation of the proposed solution has been verified in terms of feature matching and scene rendering and compared comprehensively with widely used methods. The experimental results demonstrate that the proposed solution can provide reliable feature matches for aerial and ground images with an obvious increase in the number of initial and refined matches, and it can provide enough matches to achieve accurate ISfM reconstruction and complete 3DGS-based scene rendering.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。