通过分布映射提升文本生成3D质量，解决色彩过饱和与过度平滑问题。
DreamMapping: High-Fidelity Text-to-3D Generation via Variational Distribution Mapping
- 将渲染图像视为扩散生成的退化实例，跳过雅可比计算加速分布建模。
- 结合时步依赖的系数衰减策略，显著提升生成细节精度。
- 适配高斯点云表示，适合追求真实感与高效生成的3D内容创作者。
Score Distillation Sampling (SDS) 已成为文本到3D生成的主流技术，通过从文本到2D引导中提炼视图依赖信息来生成3D内容。然而，该方法常出现色彩过饱和和过度平滑等问题。本文对SDS进行深入分析，发现其核心在于建模渲染图像的分布。基于此洞察，提出一种新策略——变分分布映射（VDM），将渲染图像视为基于扩散生成的退化实例，从而跳过扩散U-Net中的雅可比计算，高效训练变分分布。同时引入时步依赖的分布系数退火（DCA）以进一步提升蒸馏精度。结合VDM与DCA，采用高斯点云（Gaussian Splatting）作为3D表示，构建了文本到3D生成框架。大量实验表明，该方法在优化效率与生成保真度方面均表现出色，能生成高保真、逼真的3D资产。
原文摘要 · Abstract (English)
Score Distillation Sampling (SDS) has emerged as a prevalent technique for text-to-3D generation, enabling 3D content creation by distilling view-dependent information from text-to-2D guidance. However, they frequently exhibit shortcomings such as over-saturated color and excess smoothness. In this paper, we conduct a thorough analysis of SDS and refine its formulation, finding that the core design is to model the distribution of rendered images. Following this insight, we introduce a novel strategy called Variational Distribution Mapping (VDM), which expedites the distribution modeling process by regarding the rendered images as instances of degradation from diffusion-based generation. This special design enables the efficient training of variational distribution by skipping the calculations of the Jacobians in the diffusion U-Net. We also introduce timestep-dependent Distribution Coefficient Annealing (DCA) to further improve distilling precision. Leveraging VDM and DCA, we use Gaussian Splatting as the 3D representation and build a text-to-3D generation framework. Extensive experiments and evaluations demonstrate the capability of VDM and DCA to generate high-fidelity and realistic assets with optimization efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。