用3D扩散模型先验，一次生成最优视角，提升物体重建效率
DM-OSVP++: One-Shot View Planning Using 3D Diffusion Models for Active RGB-Based Object Reconstruction
- 基于初始多视角图像，用3D扩散模型生成物体初模作为规划基础
- 融合几何与纹理分布，自动聚焦复杂区域，生成高信息量视角
- 支持单次规划，适用于需快速重建的机器人场景
主动物体重建在众多机器人应用中至关重要。关键在于生成针对特定物体的观测配置，以获取有助于重建的信息。一次性视图规划可在无需耗时在线重规划的情况下，实现高效数据采集。本文核心洞察是利用3D扩散模型的生成能力作为有价值的先验信息。通过条件化于初始多视角图像，我们利用3D扩散模型中的先验生成近似物体模型，作为视图规划的基础。所提新方法将物体模型的几何与纹理分布融入视图规划过程，生成聚焦于待重建物体复杂部分的视图。通过仿真与真实世界实验验证了该主动物体重建系统的有效性，证明了使用3D扩散先验进行一次性视图规划的可行性与优越性。
原文摘要 · Abstract (English)
Active object reconstruction is crucial for many robotic applications. A key aspect in these scenarios is generating object-specific view configurations to obtain informative measurements for reconstruction. One-shot view planning enables efficient data collection by predicting all views at once, eliminating the need for time-consuming online replanning. Our primary insight is to leverage the generative power of 3D diffusion models as valuable prior information. By conditioning on initial multi-view images, we exploit the priors from the 3D diffusion model to generate an approximate object model, serving as the foundation for our view planning. Our novel approach integrates the geometric and textural distributions of the object model into the view planning process, generating views that focus on the complex parts of the object to be reconstructed. We validate the proposed active object reconstruction system through both simulation and real-world experiments, demonstrating the effectiveness of using 3D diffusion priors for one-shot view planning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。