提出一种保持扩散模型潜空间分布的推理时引导方法
DiffRGD: An Inference-Time Diffusion Guidance Through Riemannian Gradient Descent
- 将采样步骤建模为球面流形上的约束优化，用黎曼梯度下降求解
- 在图像修复和条件生成任务中，样本质量优于现有方法
- 无需重训练，可直接接入任意预训练扩散模型
扩散模型广泛应用于生成建模，成为诸多图像生成任务的基础。为在不进行昂贵重训练或微调的情况下控制生成过程，许多研究探索推理时引导方法，通过可微目标在推理阶段调整潜在表示。然而，这些方法因引入分布漂移，无法有效保持原始高斯分布，导致样本质量下降。为此，本文提出 DiffRGD，一种分布感知的引导框架，显式保留潜在空间的高斯结构。DiffRGD 将每个采样步骤建模为由潜在高斯分布诱导的球面流形上的约束优化问题，并通过黎曼梯度下降（Riemannian Gradient Descent, RGD）高效求解。该方法为即插即用型，可无缝集成至任意预训练扩散模型。大量实验表明，DiffRGD 在多数图像修复与条件生成任务中优于先前方法。
原文摘要 · Abstract (English)
Recently, diffusion models have been widely adopted in generative modeling and have served as foundational models for many image generation tasks. To control the generation without costly re-training or fine-tuning, many works seek inference-time guidance methods to steer the latent via a differentiable objective at inference time. However, these methods cannot effectively preserve the original Gaussian distribution because they introduce distributional drift, thereby degrading the sample quality. To address this gap, we propose DiffRGD, a distribution-aware guidance framework that explicitly preserves the latent Gaussian structure. DiffRGD formulates each sampling step as a constrained optimization problem on a spherical manifold induced by the latent Gaussian distribution, and solves it efficiently via Riemannian Gradient Descent (RGD). DiffRGD is a plug-and-play method that can be seamlessly integrated into any pre-trained diffusion model. Extensive experiments demonstrate that DiffRGD outperforms previous methods in most image restoration and conditional generation tasks. Our project page is available at https://diffrgd.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。