无需掩码实现遮挡下多物体完整3D重建，支持物理仿真交互
Occlusion-Robust Multi-Object Decoupling for Physics-Based Robotic Interaction
- 基于稀疏视角重建，用扩散模型联合优化新视角合成与纹理保真
- 在合成、机器人和真实数据集上均生成完整可仿真的3D物体
- 适合需要真实物理交互的机器人场景重建任务
我们提出一种无掩码方法，从稀疏且遮挡的现实视角中实现无损多物体3D重建，支持通过材料点法（MPM）模拟进行物理上合理的机器人交互。核心洞察是物体耦合源于遮挡和视角有限，因此将多物体解耦建模为稀疏视图重建问题。采用3D高斯泼溅作为基础表示，首先利用SAM2训练的分割场获得粗粒度实例划分。不依赖掩码，而是通过联合得分蒸馏采样（SDS）过程重建断裂几何体，该过程融合参考视图监督与由2D/3D扩散先验引导的新视角合成，以确保纹理一致性和三维一致性。此外，引入几何感知先验，如物体内部与物体间相似性，以规范几何推理。实验表明，该方法无需人工掩码即可生成完整、可仿真交互的3D物体，在合成、机器人及真实世界数据集上均表现优异。
原文摘要 · Abstract (English)
We propose a mask-free method for lossless multi-object 3D reconstruction from sparse and occluded real-world views, enabling physically plausible robotic interaction via Material Point Method (MPM) simulation. Our key insight is that object coupling stems from occlusion and limited viewpoints, which we address by formulating multi-object decoupling as a sparse-view reconstruction problem. Using 3D Gaussian Splatting as base representation, we first obtain coarse instance partitions with a SAM2-trained segmentation field. Rather than relying on masks, we reconstruct fragmented geometries by leveraging a joint Score Distillation Sampling (SDS) process, which integrates reference-view supervision with novel-view synthesis guided by 2D and 3D diffusion priors to enforce both texture fidelity and 3D consistency. Furthermore, we incorporate geometry-aware priors such as intra-object and inter-object similarity to regularize geometric reasoning. Experimental results demonstrate that our method produces complete, simulation-ready 3D objects without requiring manual masks, enabling realistic dynamic interactions on both synthetic, robotic and real-world datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。