arXiv:2412.06780cs.CV2024-12

让3D生成更多样,用随机种子引导优化路径

Diverse Score Distillation

  • 用扩散模型采样路径的随机种子引导3D优化
  • 在文本到3D和单视图重建中显著提升多样性
  • 适合需要丰富输出的3D生成任务

2D扩散模型的得分蒸馏已被证明是引导3D优化的强大机制,例如实现基于文本的3D生成或单视图重建。然而,现有得分蒸馏方法的一个常见局限是,尽管底层扩散模型能生成多样化样本,优化结果却缺乏多样性。受去噪扩散采样过程启发,本文提出一种新的得分形式,通过随机初始种子定义生成路径,从而确保多样性。针对优化可能无法精确遵循生成路径的情况(如渲染相互依赖的3D表示),我们进一步提出近似方法。我们在2D优化、基于文本的3D推理和单视图重建等任务上展示了所提Diverse Score Distillation(DSD)的有效性。实验证明,DSD相比先前方法显著提升了样本多样性,同时保持了高保真度。

原文摘要 · Abstract (English)

Score distillation of 2D diffusion models has proven to be a powerful mechanism to guide 3D optimization, for example enabling text-based 3D generation or single-view reconstruction. A common limitation of existing score distillation formulations, however, is that the outputs of the (mode-seeking) optimization are limited in diversity despite the underlying diffusion model being capable of generating diverse samples. In this work, inspired by the sampling process in denoising diffusion, we propose a score formulation that guides the optimization to follow generation paths defined by random initial seeds, thus ensuring diversity. We then present an approximation to adopt this formulation for scenarios where the optimization may not precisely follow the generation paths (\eg a 3D representation whose renderings evolve in a co-dependent manner). We showcase the applications of our `Diverse Score Distillation' (DSD) formulation across tasks such as 2D optimization, text-based 3D inference, and single-view reconstruction. We also empirically validate DSD against prior score distillation formulations and show that it significantly improves sample diversity while preserving fidelity.

3D生成扩散模型多样性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。