解决双物体交互时的3D分离重建难题
Betsu-Betsu: Multi-View Separable 3D Reconstruction of Two Interacting Objects
- 基于神经隐式表示，实现双物体在交互中的清晰3D分离
- 在极端遮挡下仍能保持几何不穿透，新视角合成效果优
- 适用于刚体与关节物体，适合人体-物体交互场景
从多视角RGB图像中进行多物体的可分离3D重建——即为两个物体生成独立的3D形状并明确区分——仍是研究较少的问题。其挑战源于物体间严重相互遮挡及交互边界处的模糊性。本文提出一种新的神经隐式方法，能够在物体紧密交互时重建其几何与外观，并在3D空间中实现有效分离，避免表面穿插，支持新视角合成。该框架端到端可训练，采用新颖的α混合正则化监督，确保在极端遮挡下两几何体仍保持良好分离。方法无需标记，适用于刚体与关节物体。我们构建了一个包含人与物体近距离交互的新数据集，并在两个武术动作场景上进行评估。实验表明，本方法在3D重建和新视角合成指标上显著优于现有可比方法。
原文摘要 · Abstract (English)
Separable 3D reconstruction of multiple objects from multi-view RGB images -- resulting in two different 3D shapes for the two objects with a clear separation between them -- remains a sparsely researched problem. It is challenging due to severe mutual occlusions and ambiguities along the objects' interaction boundaries. This paper investigates the setting and introduces a new neuro-implicit method that can reconstruct the geometry and appearance of two objects undergoing close interactions while disjoining both in 3D, avoiding surface inter-penetrations and enabling novel-view synthesis of the observed scene. The framework is end-to-end trainable and supervised using a novel alpha-blending regularisation that ensures that the two geometries are well separated even under extreme occlusions. Our reconstruction method is markerless and can be applied to rigid as well as articulated objects. We introduce a new dataset consisting of close interactions between a human and an object and also evaluate on two scenes of humans performing martial arts. The experiments confirm the effectiveness of our framework and substantial improvements using 3D and novel view synthesis metrics compared to several existing approaches applicable in our setting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。