单图恢复带真实尺度的精细3D场景，精度远超以往方法。
MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
- 基于仿射不变点云，通过新策略实现真实尺度重建。
- 在多个数据集上同时提升相对几何精度与细节还原能力。
- 适合需要高保真3D重建的视觉、自动驾驶与机器人应用。
我们提出MoGe-2，一种先进的单目几何估计模型，可从单张图像恢复具有真实尺度的3D点云。该方法在近期的单目几何估计方法MoGe基础上改进，后者仅能预测无尺度的仿射不变点云。我们探索有效策略，在不牺牲仿射不变点表示提供的相对几何精度的前提下，实现真实尺度的几何重建。此外，我们发现真实数据中的噪声和误差会削弱预测几何的细粒度细节。为此，我们设计了一种统一的数据精炼方法，利用高精度合成标签对来自不同来源的真实数据进行过滤与补全，显著提升了重建几何的细节层次，同时保持整体精度。我们在大规模混合数据集上训练模型，并进行了全面评估，结果表明其在相对几何准确性、真实尺度精度以及细粒度细节恢复方面均表现卓越，是首个同时实现这三项能力的方法。
原文摘要 · Abstract (English)
We propose MoGe-2, an advanced open-domain geometry estimation model that recovers a metric scale 3D point map of a scene from a single image. Our method builds upon the recent monocular geometry estimation approach, MoGe, which predicts affine-invariant point maps with unknown scales. We explore effective strategies to extend MoGe for metric geometry prediction without compromising the relative geometry accuracy provided by the affine-invariant point representation. Additionally, we discover that noise and errors in real data diminish fine-grained detail in the predicted geometry. We address this by developing a unified data refinement approach that filters and completes real data from different sources using sharp synthetic labels, significantly enhancing the granularity of the reconstructed geometry while maintaining the overall accuracy. We train our model on a large corpus of mixed datasets and conducted comprehensive evaluations, demonstrating its superior performance in achieving accurate relative geometry, precise metric scale, and fine-grained detail recovery -- capabilities that no previous methods have simultaneously achieved.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。