用生成模型补全视角缺失的3D物体,重建更完整准确。
ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generation
- 将重建先验融入生成框架,强化多视角特征关联
- 在全局结构与局部细节上均实现高一致性重建
- 适合处理遮挡多、视角稀疏的3D重建场景
现有多视角3D物体重建方法严重依赖输入视角间的充分重叠,实际中因遮挡和覆盖稀疏导致重建严重不完整。基于扩散的3D生成技术通过学习先验来推测不可见部分,有望解决此问题。然而,其推理过程的随机性限制了生成结果的准确性和可靠性,使现有框架难以整合此类生成先验。本文深入分析扩散式3D生成方法一致性差的原因:(a) 多视角图像特征提取时跨视角连接构建不足;(b) 迭代去噪过程中局部细节生成可控性差,易产生与输入不一致的几何与纹理细节。为此,提出ReconViaGen,创新性地将重建先验嵌入生成框架,并设计多种策略有效解决上述问题。大量实验表明,ReconViaGen可在全局结构与局部细节上均实现与输入视图高度一致的完整且精确的3D重建。
原文摘要 · Abstract (English)
Existing multi-view 3D object reconstruction methods heavily rely on sufficient overlap between input views, where occlusions and sparse coverage in practice frequently yield severe reconstruction incompleteness. Recent advancements in diffusion-based 3D generative techniques offer the potential to address these limitations by leveraging learned generative priors to hallucinate invisible parts of objects, thereby generating plausible 3D structures. However, the stochastic nature of the inference process limits the accuracy and reliability of generation results, preventing existing reconstruction frameworks from integrating such 3D generative priors. In this work, we comprehensively analyze the reasons why diffusion-based 3D generative methods fail to achieve high consistency, including (a) the insufficiency in constructing and leveraging cross-view connections when extracting multi-view image features as conditions, and (b) the poor controllability of iterative denoising during local detail generation, which easily leads to plausible but inconsistent fine geometric and texture details with inputs. Accordingly, we propose ReconViaGen to innovatively integrate reconstruction priors into the generative framework and devise several strategies that effectively address these issues. Extensive experiments demonstrate that our ReconViaGen can reconstruct complete and accurate 3D models consistent with input views in both global structure and local details.Project page: https://jiahao620.github.io/reconviagen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。