用监督学习实现高效可控扩散生成，无需强化学习。
Efficient Controllable Diffusion via Optimal Classifier Guidance
- 基于在线数据迭代训练小分类器引导生成过程。
- 生成质量高，推理时间接近基础模型。
- 适用于图像与生物序列生成，理论可证明收敛性。
可控扩散模型的生成目标是引导模型生成优化特定目标函数的样本，广泛应用于图像、分子及DNA/序列生成。现有基于强化学习的微调方法易过拟合且资源消耗大。本文将可控生成建模为在KL正则化目标下的分布优化问题，提出SLCD——基于监督学习的可控扩散方法：通过迭代生成在线数据并训练小型分类器来引导扩散模型生成。与标准分类器引导扩散类似，其核心计算为分类，不涉及复杂强化学习或控制概念。通过归约为无后悔在线学习分析，证明在KL散度下SLCD输出可保证收敛至最优解。实验证明，SLCD在连续扩散图像生成和离散扩散生物序列生成中均能生成高质量样本，推理时间几乎与基础模型相当。代码已开源。
原文摘要 · Abstract (English)
The controllable generation of diffusion models aims to steer the model to generate samples that optimize some given objective functions. It is desirable for a variety of applications including image generation, molecule generation, and DNA/sequence generation. Reinforcement Learning (RL) based fine-tuning of the base model is a popular approach but it can overfit the reward function while requiring significant resources. We frame controllable generation as a problem of finding a distribution that optimizes a KL-regularized objective function. We present SLCD -- Supervised Learning based Controllable Diffusion, which iteratively generates online data and trains a small classifier to guide the generation of the diffusion model. Similar to the standard classifier-guided diffusion, SLCD's key computation primitive is classification and does not involve any complex concepts from RL or control. Via a reduction to no-regret online learning analysis, we show that under KL divergence, the output from SLCD provably converges to the optimal solution of the KL-regularized objective. Further, we empirically demonstrate that SLCD can generate high quality samples with nearly the same inference time as the base model in both image generation with continuous diffusion and biological sequence generation with discrete diffusion. Our code is available at https://github.com/Owen-Oertell/slcd
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。