用预训练扩散模型无训练采样高质量可微表示，提升图像与3D生成质量。
Diffusing Differentiable Representations
- 将扩散模型逆向过程回溯到可微表示参数空间，直接更新参数采样。
- 生成的可微表示在图像、全景和3D NeRF上质量与多样性显著优于现有方法。
- 无需训练，通用性强，适合需高保真生成的视觉任务研究者。
我们提出一种全新的、无需训练的可微表示（diffrep）采样方法，利用预训练扩散模型实现。该方法不局限于寻找模式，而是将反向过程的动力学“回溯”至diffrep参数空间，并根据回溯过程更新参数。我们发现diffrep样本隐含一个约束条件，解决该约束可显著提升生成物体的一致性与细节。相比现有技术，本方法在图像、全景图和3D NeRF上均生成了质量与多样性显著更高的可微表示。该方法为diffrep采样提供通用方案，拓展了扩散模型可解决的问题范围。
原文摘要 · Abstract (English)
We introduce a novel, training-free method for sampling differentiable representations (diffreps) using pretrained diffusion models. Rather than merely mode-seeking, our method achieves sampling by "pulling back" the dynamics of the reverse-time process--from the image space to the diffrep parameter space--and updating the parameters according to this pulled-back process. We identify an implicit constraint on the samples induced by the diffrep and demonstrate that addressing this constraint significantly improves the consistency and detail of the generated objects. Our method yields diffreps with substantially improved quality and diversity for images, panoramas, and 3D NeRFs compared to existing techniques. Our approach is a general-purpose method for sampling diffreps, expanding the scope of problems that diffusion models can tackle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。