用3D空间对齐解决全景拼接畸变问题
Pano360: Perspective to Panoramic Vision with Geometric Consistency
- 将2D拼接转为3D空间全局对齐，利用相机位姿指导图像变换
- 在真实场景数据集上实现更优的对齐精度与视觉质量
- 适合需要高精度全景图像的应用，如虚拟现实和测绘
现有全景拼接方法严重依赖成对特征匹配，难以利用多视图间的几何一致性，导致在弱纹理、大视差和重复图案等复杂场景中出现严重畸变和错位。由于多视图几何对应关系可在三维空间中直接构建，更具准确性和全局一致性，本文将二维对齐任务拓展至三维摄影测量空间。采用新型基于Transformer的架构实现三维感知，并聚合所有视角的全局信息。该方法直接利用相机位姿引导图像扭曲，实现三维空间中的全局对齐，并通过多特征联合优化策略计算拼接缝。此外，为训练和评估网络，我们构建了一个大规模真实场景数据集。大量实验表明，该方法在对齐精度和视觉质量上显著优于现有方法。
原文摘要 · Abstract (English)
Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。