用多视角扩散先验解决3D生成的几何不一致问题
Bridging Geometry-Coherent Text-to-3D Generation with Multi-View Diffusion Priors and Gaussian Splatting
- 将多视角联合分布引入优化，耦合不同视角先验
- 直接优化3D高斯点云，生成几何一致的3D内容
- 适合需要高质量3D生成与几何保真的研究者
Score Distillation Sampling (SDS) 利用预训练2D扩散模型推动文本到3D生成，但忽略了多视角相关性，易产生几何不一致和多脸伪影。本文提出耦合得分蒸馏（CSD）框架，通过引入多视角联合分布先验，确保几何一致性的同时实现3D高斯点云的稳定直接优化。具体地，将优化重构为多视角联合优化问题，导出有效优化规则，跨视角耦合先验并保持生成资产多样性。此外，提出直接优化随机初始化的3D高斯点云（3D-GS）以生成几何一致的3D内容，并利用可变形四面体网格从3D-GS初始化，经由CSD精炼生成高质量精细网格。定量与定性实验表明该方法高效且具备竞争力。
原文摘要 · Abstract (English)
Score Distillation Sampling (SDS) leverages pretrained 2D diffusion models to advance text-to-3D generation but neglects multi-view correlations, being prone to geometric inconsistencies and multi-face artifacts in the generated 3D content. In this work, we propose Coupled Score Distillation (CSD), a framework that couples multi-view joint distribution priors to ensure geometrically consistent 3D generation while enabling the stable and direct optimization of 3D Gaussian Splatting. Specifically, by reformulating the optimization as a multi-view joint optimization problem, we derive an effective optimization rule that effectively couples multi-view priors to guide optimization across different viewpoints while preserving the diversity of generated 3D assets. Additionally, we propose a framework that directly optimizes 3D Gaussian Splatting (3D-GS) with random initialization to generate geometrically consistent 3D content. We further employ a deformable tetrahedral grid, initialized from 3D-GS and refined through CSD, to produce high-quality, refined meshes. Quantitative and qualitative experimental results demonstrate the efficiency and competitive quality of our approach.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。