用反演方法让扩散模型做目标检测，速度更快精度不降
DIVE: Inverting Conditional Diffusion Models for Discriminative Tasks
- 通过梯度优化替代传统枚举，加速推理过程
- 在COCO数据集上达到基础检测模型水平
- 适合需要快速部署的生成式模型应用者
扩散模型在图像、视频生成等生成任务中取得显著进展。本文研究如何利用预训练的扩散模型完成判别任务。具体地，通过“反演”预训练的布局到图像扩散模型，将预训练冻结生成模型的判别能力从分类任务扩展到更复杂的物体检测任务。为此，分别提出基于梯度的离散优化方法以替代耗时的预测枚举过程，以及先验分布模型以更准确地利用贝叶斯规则。实验结果表明，该方法在COCO数据集上的表现与基础判别性目标检测基线相当。此外，本方法在不牺牲准确率的前提下，大幅提升了此前基于扩散模型的分类方法的速度。代码与模型已开源：https://github.com/LiYinqi/DIVE。
原文摘要 · Abstract (English)
Diffusion models have shown remarkable progress in various generative tasks such as image and video generation. This paper studies the problem of leveraging pretrained diffusion models for performing discriminative tasks. Specifically, we extend the discriminative capability of pretrained frozen generative diffusion models from the classification task to the more complex object detection task, by "inverting" a pretrained layout-to-image diffusion model. To this end, a gradient-based discrete optimization approach for replacing the heavy prediction enumeration process, and a prior distribution model for making more accurate use of the Bayes' rule, are proposed respectively. Empirical results show that this method is on par with basic discriminative object detection baselines on COCO dataset. In addition, our method can greatly speed up the previous diffusion-based method for classification without sacrificing accuracy. Code and models are available at https://github.com/LiYinqi/DIVE .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。