用2D扩散模型补全3D点云，高效生成细节丰富的完整形状。
Repurposing 2D Diffusion Models for 3D Shape Completion
- 构建2D几何表示'形状图谱'，打通3D输入与2D生成空间的模态鸿沟。
- 在PCN和ShapeNet-55上实现高质量补全，细节保留优于现有方法。
- 适合需要低数据量、高保真3D重建的工业设计与艺术创作场景。
我们提出一个框架，将2D扩散模型用于从不完整点云中完成3D形状。尽管文本到图像扩散模型因丰富2D数据取得了显著成功，但3D扩散模型因高质量3D数据稀缺及3D输入与2D隐空间间的模态差异而发展滞后。为克服这些限制，我们引入了形状图谱（Shape Atlas），一种紧凑的3D几何2D表示，它（1）充分利用预训练2D扩散模型的生成能力，（2）对齐条件输入与输出空间的模态，实现更有效的条件生成。这种统一的2D形式使模型能从有限3D数据中学习，并生成高质量、细节保留的形状补全结果。我们在PCN和ShapeNet-55数据集上验证了方法的有效性。此外，我们展示了从补全点云生成艺术家创建网格的下游应用，进一步证明了该方法的实用性。
原文摘要 · Abstract (English)
We present a framework that adapts 2D diffusion models for 3D shape completion from incomplete point clouds. While text-to-image diffusion models have achieved remarkable success with abundant 2D data, 3D diffusion models lag due to the scarcity of high-quality 3D datasets and a persistent modality gap between 3D inputs and 2D latent spaces. To overcome these limitations, we introduce the Shape Atlas, a compact 2D representation of 3D geometry that (1) enables full utilization of the generative power of pretrained 2D diffusion models, and (2) aligns the modalities between the conditional input and output spaces, allowing more effective conditioning. This unified 2D formulation facilitates learning from limited 3D data and produces high-quality, detail-preserving shape completions. We validate the effectiveness of our results on the PCN and ShapeNet-55 datasets. Additionally, we show the downstream application of creating artist-created meshes from our completed point clouds, further demonstrating the practicality of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。