用360°视频构建真实世界相机位姿评估基准，挑战现有算法
ORBIT++: Benchmarking SfM in the Wild with 360° Video
- 利用在线全景360°视频生成带真实轨迹的挑战性片段
- 多款主流SfM方法在新基准上表现不佳，误差显著升高
- 适合研究真实场景下三维重建与位姿估计的学者使用
结构光重建（SfM）是三维感知的核心技术，但现有方法在复杂视频中常因剧烈运动或动态场景而失效。同时，该领域缺乏针对此类困难情况的可靠真值基准，难以衡量实际进展或定位改进方向。为此，我们提出一种新基准，利用在线全景360°视频作为数据源，在保留可恢复真值轨迹的同时生成高难度测试片段。由于全景特性提供了更丰富的视觉上下文，即使部分画面模糊、运动或存在动态物体，仍能有效追踪相机运动。通过全向视频追踪后裁剪并重投影，生成透视视角片段作为基准，命名为ORBIT。实验表明，COLMAP及近年基于优化和前馈的SfM方法在该基准上均出现显著位姿误差。因此，ORBIT为研究人员提供了一个真实世界级的评测平台，可用于衡量真正挑战性场景下的算法性能。
原文摘要 · Abstract (English)
Structure-from-Motion (SfM) is a cornerstone of 3D perception, yet current methods often fail when applied to complex videos involving challenging camera motions or dynamic scenes. Compounding the problem, the field lacks reliable ground-truth benchmarks for such difficult scenarios, making it hard to gauge real-world progress or to pinpoint where improvements are most needed. To address this gap, we introduce a new benchmark for evaluating camera pose estimation. Our key insight is to leverage online panoramic 360° video as a source of data from which to construct challenging clips, while still enabling robust ground-truth trajectory recovery. The panoramic nature of these videos provides richer visual context for tracking camera motion, even when parts of the view are affected by blur, motion, or dynamic objects. After tracking camera motion across full 360° videos, we crop and reproject selected portions to generate perspective-view clips that serve as our benchmark, called ORBIT. Experiments show that COLMAP, as well as recent optimization-based and feed-forward SfM methods struggle to accurately estimate camera poses on our benchmark. Hence, ORBIT provides a valuable testbed where researchers can meaningfully measure progress on truly challenging, real-world SfM problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。