3秒内完成3D物体编辑，无需复杂优化。
Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects
- 将3D编辑转为多视角图像修复问题，通用性强。
- 仅需3秒即可完成编辑，质量优于此前方法。
- 适合需要快速迭代的3D内容创作者使用。
我们提出一种生成式技术，可在约3秒内编辑以网格、NeRF或高斯点云表示的3D形状，无需运行类似SDS的优化过程。核心思路是将3D编辑视为多视角图像修复问题，该表示形式通用，可通过现有的大型重建模型映射回任意3D表示。我们探索了多种微调策略，使同一扩散模型同时具备多视角生成与修复能力。特别是，修复掩码的设计对训练修复模型至关重要，我们提出了几种掩码策略，以模拟用户对3D形状的实际编辑操作。该方法将3D生成编辑时间从数小时缩短至3秒，且结果质量优于先前工作。
原文摘要 · Abstract (English)
We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。