用透视变换实现高效多视角物体移除,速度提升至五分之一。
HOMER: Homography-Based Efficient Multi-view 3D Object Removal
- 通过区域交互与透视映射,高效生成多视角物体掩码。
- 仅在关键视图上进行修复,计算量减少至基线的1/5。
- 兼容NeRF、3DGS等框架,适合实际场景应用。
3D物体移除是3D场景编辑的重要任务,广泛应用于场景理解、增强现实和机器人领域。然而,现有方法在多视角设置下难以兼顾一致性、可用性与计算效率。主要瓶颈在于源视角交互不直观、多视角掩码生成低效、修复过程计算昂贵,且对不同辐射场表示适配性差。为此,我们提出一种新流程:在源视角采用直观的区域交互机制,无需相机位姿或额外训练。引入轻量级HoMM模块实现高质量多视角掩码传播。在修复阶段,仅对选定的关键视图进行修复,并通过基于透视变换的映射将结果传播至其他视图,显著降低计算开销。该方法兼容NeRF、3D Gaussian Splatting等多种辐射场框架,展现出更强通用性与实用性。此外,我们构建了一个新的3D多物体移除数据集,包含更丰富的物体类别和视角变化。在公开基准和自建数据集上的实验表明,本方法达到当前最优性能,同时推理时间仅为领先基线的1/5。
原文摘要 · Abstract (English)
3D object removal is an important sub-task in 3D scene editing, with broad applications in scene understanding, augmented reality, and robotics. However, existing methods struggle to achieve a desirable balance among consistency, usability, and computational efficiency in multi-view settings. These limitations are primarily due to unintuitive user interaction in the source view, inefficient multi-view object mask generation, computationally expensive inpainting procedures, and a lack of applicability across different radiance field representations. To address these challenges, we propose a novel pipeline that improves the quality and efficiency of multi-view object mask generation and inpainting. Our method introduces an intuitive region-based interaction mechanism in the source view and eliminates the need for camera poses or extra model training. Our lightweight HoMM module is employed to achieve high-quality multi-view mask propagation with enhanced efficiency. In the inpainting stage, we further reduce computational costs by performing inpainting only on selected key views and propagating the results to other views via homography-based mapping. Our pipeline is compatible with a variety of radiance field frameworks, including NeRF and 3D Gaussian Splatting, demonstrating improved generalizability and practicality in real-world scenarios. Additionally, we present a new 3D multi-object removal dataset with greater object diversity and viewpoint variation than existing datasets. Experiments on public benchmarks and our proposed dataset show that our method achieves state-of-the-art performance while reducing runtime to one-fifth of that required by leading baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。