仅用单张图像实现高精度3D平面重建,通用性强且鲁棒。
MonoPlane: Exploiting Monocular Geometric Cues for Generalizable 3D Plane Reconstruction
- 基于单图深度与法向量,用图引导RANSAC逐个拟合平面。
- 在多个数据集上零样本迁移表现领先,平面重建精度显著提升。
- 适合需要轻量化、跨场景3D建模的开发者与研究者使用。
本文提出名为MonoPlane的通用3D平面检测与重建框架。不同于依赖多视图或RGB-D输入的鲁棒估计方法,以及易受域偏移影响的学习方法,MonoPlane结合两者优势,基于单目几何线索建立可泛化的3D平面重建流程,在真实复杂场景中实现准确、鲁棒且可扩展的平面重建。首先利用大规模预训练神经网络从单张图像获取深度和表面法向量;随后将这些单目几何线索融入邻近引导的RANSAC框架,依次拟合每个平面实例。通过图结构建模3D点邻近性,指导从噪声单目深度中进行平面拟合,并引入图像级多平面联合优化以提升所有平面实例间的一致性。进一步设计简洁高效的流程,将单视图方案拓展至稀疏视图3D平面重建。大量实验表明,该方法在多个数据集上均展现出优越的零样本泛化能力,转移设置下达到当前最优的平面重建性能。代码已开源:https://github.com/thuzhaowang/MonoPlane。
原文摘要 · Abstract (English)
This paper presents a generalizable 3D plane detection and reconstruction framework named MonoPlane. Unlike previous robust estimator-based works (which require multiple images or RGB-D input) and learning-based works (which suffer from domain shift), MonoPlane combines the best of two worlds and establishes a plane reconstruction pipeline based on monocular geometric cues, resulting in accurate, robust and scalable 3D plane detection and reconstruction in the wild. Specifically, we first leverage large-scale pre-trained neural networks to obtain the depth and surface normals from a single image. These monocular geometric cues are then incorporated into a proximity-guided RANSAC framework to sequentially fit each plane instance. We exploit effective 3D point proximity and model such proximity via a graph within RANSAC to guide the plane fitting from noisy monocular depths, followed by image-level multi-plane joint optimization to improve the consistency among all plane instances. We further design a simple but effective pipeline to extend this single-view solution to sparse-view 3D plane reconstruction. Extensive experiments on a list of datasets demonstrate our superior zero-shot generalizability over baselines, achieving state-of-the-art plane reconstruction performance in a transferring setting. Our code is available at https://github.com/thuzhaowang/MonoPlane .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。