用可见性分割预训练，提升相机位姿估计精度
Alligat0R: Pre-Training Through Co-Visibility Segmentation for Relative Camera Pose Regression
- 以像素级可见性预测替代跨视图补全，增强预训练效果
- 在nuScenes和ScanNet基础上构建500万图像对数据集，覆盖多样重叠场景
- 相比CroCo显著提升相对位姿回归性能，结果可解释性强
预训练技术极大推动了计算机视觉发展，其中CroCo的跨视图补全方法在3D重建和位姿回归任务中表现优异。然而，该方法在非共视区域存在病态问题，限制其效果。本文提出Alligat0R，一种新颖的预训练方法，将跨视图学习替换为共视性分割任务：预测某图像中每个像素是否在另一图像中共视、被遮挡或超出视野。该方法在共视与非共视区域均有效，且提供可解释的预测结果。为此，我们构建了Cub3数据集，包含500万图像对及基于nuScenes和ScanNet的密集共视性标注，涵盖多种重叠程度的场景。实验表明，Alligat0R在相对位姿回归任务上显著优于CroCo。代码已公开于https://github.com/thibautloiseau/alligat0r。
原文摘要 · Abstract (English)
Pre-training techniques have greatly advanced computer vision, with CroCo's cross-view completion approach yielding impressive results in tasks like 3D reconstruction and pose regression. However, cross-view completion is ill-posed in non-covisible regions, limiting its effectiveness. We introduce Alligat0R, a novel pre-training approach that replaces cross-view learning with a covisibility segmentation task. Our method predicts whether each pixel in one image is covisible in the second image, occluded, or outside the field of view, making the pre-training effective in both covisible and non-covisible regions, and provides interpretable predictions. To support this, we present Cub3, a large-scale dataset with 5M image pairs and dense covisibility annotations derived from the nuScenes and ScanNet datasets. Cub3 includes diverse scenarios with varying degrees of overlap. The experiments show that our novel pre-training method Alligat0R significantly outperforms CroCo in relative pose regression. Code is available at https://github.com/thibautloiseau/alligat0r.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。