用JS散度改进文本生成3D模型，解决过饱和与多样性不足问题。
Text-to-3D Generation using Jensen-Shannon Score Distillation
- 基于JS散度设计新目标函数，稳定优化过程
- 在T3Bench上生成高质量且多样化的3D资产
- 适合需要高保真与多样性的3D生成研究者
Score distillation sampling 是一种利用预训练大规模文本到图像扩散模型指导生成3D模型的有效技术。然而，生成的3D资产常出现过饱和、过度平滑且多样性有限的问题，这源于反向Kullback-Leibler散度目标导致优化不稳定和模式追逐行为。本文提出基于Jensen-Shannon散度（JSD）的有界评分蒸馏目标，该方法稳定优化过程并有效缓解模式追逐。通过生成对抗网络理论，假设判别器已充分训练，构建近似目标函数；进一步假设判别器为对数几率分类器，提出少数样本采样算法以估计梯度，实现可实践的JSD方案。理论与实证分析验证了该方法的有效性。在T3Bench上的实验表明，该方法能生成高质量且多样化的3D资产。
原文摘要 · Abstract (English)
Score distillation sampling is an effective technique to generate 3D models from text prompts, utilizing pre-trained large-scale text-to-image diffusion models as guidance. However, the produced 3D assets tend to be over-saturating, over-smoothing, with limited diversity. These issues are results from a reverse Kullback-Leibler (KL) divergence objective, which makes the optimization unstable and results in mode-seeking behavior. In this paper, we derive a bounded score distillation objective based on Jensen-Shannon divergence (JSD), which stabilizes the optimization process and produces high-quality 3D generation. JSD can match well generated and target distribution, therefore mitigating mode seeking. We provide a practical implementation of JSD by utilizing the theory of generative adversarial networks to define an approximate objective function for the generator, assuming the discriminator is well trained. By assuming the discriminator following a log-odds classifier, we propose a minority sampling algorithm to estimate the gradients of our proposed objective, providing a practical implementation for JSD. We conduct both theoretical and empirical studies to validate our method. Experimental results on T3Bench demonstrate that our method can produce high-quality and diversified 3D assets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。