arXiv:2501.16737cs.CV2025-01被引 4

用扩散模型融合2D与3D先验,提升单图3D重建一致性

Consistency Diffusion Models for Single-Image 3D Reconstruction with Priors

  • 在变分贝叶斯框架中引入3D结构先验作为约束项
  • 将输入图像的2D先验投影到3D点云增强引导
  • 在合成与真实数据集上均达新基准,适合3D生成研究者

本文研究从单张图像重建3D点云的问题。目标是构建一致性扩散模型,在贝叶斯框架下协同利用2D与3D先验,确保重建过程的高一致性,这是该领域的重要挑战。提出一种新颖的扩散模型训练框架,包含两项关键创新:首先,将初始3D点云导出的3D结构先验作为边界项引入,增强变分贝叶斯框架中的证据,利用强固的内在先验严格控制扩散训练过程,提升重建一致性;其次,从单张输入图像提取2D先验,并将其投影至3D点云,丰富扩散训练的指导信息。该框架避免了直接施加额外约束带来的模型学习偏移问题,且精确实现2D先验向3D域的转换。大量实验表明,该方法在合成与真实数据集上均达到新基准。代码随提交附上。

原文摘要 · Abstract (English)

This paper delves into the study of 3D point cloud reconstruction from a single image. Our objective is to develop the Consistency Diffusion Model, exploring synergistic 2D and 3D priors in the Bayesian framework to ensure superior consistency in the reconstruction process, a challenging yet critical requirement in this field. Specifically, we introduce a pioneering training framework under diffusion models that brings two key innovations. First, we convert 3D structural priors derived from the initial 3D point cloud as a bound term to increase evidence in the variational Bayesian framework, leveraging these robust intrinsic priors to tightly govern the diffusion training process and bolster consistency in reconstruction. Second, we extract and incorporate 2D priors from the single input image, projecting them onto the 3D point cloud to enrich the guidance for diffusion training. Our framework not only sidesteps potential model learning shifts that may arise from directly imposing additional constraints during training but also precisely transposes the 2D priors into the 3D domain. Extensive experimental evaluations reveal that our approach sets new benchmarks in both synthetic and real-world datasets. The code is included with the submission.

3D重建扩散模型先验融合点云生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。