通过视角一致性优化,减少单图生成3D模型时的幻觉错误。
Dehallu3D: Hallucination-Mitigated 3D Generation from Single Image via Cyclic View Consistency Refinement
- 引入循环视角一致性约束,平衡平滑与细节保留。
- 在多个数据集上显著降低结构异常,提升3D模型几何保真度。
- 适合关注3D生成质量与真实感的应用者,如游戏和虚拟现实开发。
大型3D重建模型虽推动了虚拟现实与游戏等领域的发展,但其仍存在幻觉问题,导致生成结果出现异常孔洞或突起等结构偏差。现有方法因从稀疏多视图图像重建,视角间隙大且不连续,加剧了幻觉现象。为此,本文提出Dehallu3D,采用可插拔优化模块,包含相邻视图一致性约束以确保几何连续性,以及自适应平滑机制以保留精细特征。进一步提出异常风险度量(ORM)评估3D生成中的几何失真程度。大量实验表明,Dehallu3D在保持结构细节的同时有效消除幻觉异常,显著提升3D网格生成质量。
原文摘要 · Abstract (English)
Large 3D reconstruction models have revolutionized the 3D content generation field, enabling broad applications in virtual reality and gaming. Just like other large models, large 3D reconstruction models suffer from hallucinations as well, introducing structural outliers (e.g., odd holes or protrusions) that deviate from the input data. However, unlike other large models, hallucinations in large 3D reconstruction models remain severely underexplored, leading to malformed 3D-printed objects or insufficient immersion in virtual scenes. Such hallucinations majorly originate from that existing methods reconstruct 3D content from sparsely generated multi-view images which suffer from large viewpoint gaps and discontinuities. To mitigate hallucinations by eliminating the outliers, we propose Dehallu3D for 3D mesh generation. Our key idea is to design a balanced multi-view continuity constraint to enforce smooth transitions across dense intermediate viewpoints, while avoiding over-smoothing that could erase sharp geometric features. Therefore, Dehallu3D employs a plug-and-play optimization module with two key constraints: (i) adjacent consistency to ensure geometric continuity across views, and (ii) adaptive smoothness to retain fine details.We further propose the Outlier Risk Measure (ORM) metric to quantify geometric fidelity in 3D generation from the perspective of outliers. Extensive experiments show that Dehallu3D achieves high-fidelity 3D generation by effectively preserving structural details while removing hallucinated outliers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。