arXiv:2412.12594cs.CV2024-12被引 9

用预训练扩散模型实现快速零样本图像分类,速度提升3万倍。

A Simple and Efficient Baseline for Zero-Shot Generative Classification

  • 基于文本到图像扩散模型与DINOv2构建简单高效的分类器。
  • ImageNet上准确率提升至71.44%,单图分类仅需0.03秒。
  • 兼具高精度与实用速度,适合真实场景部署。

大型扩散模型已成为学术研究和工业AIGC应用中的主流生成模型。近期多项工作探索如何利用大型扩散模型作为零样本分类器。尽管现有基于扩散模型的零样本分类器在基准数据集上取得进展,但其分类速度极慢(如在ImageNet上单图分类需约1000秒),严重制约实际应用。本文提出一种极其简单且高效的零样本高斯扩散分类器(GDC),基于预训练文本到图像扩散模型与DINOv2。所提GDC不仅在ImageNet上超越先前方法超过10个百分点(61.40% → 71.44%),还将单图分类时间加速超过30000倍(1000秒 → 0.03秒)。此外,该方法可提供结果的概率解释。大量实验表明,GDC在多个数据集上表现优异,且能随着更强扩散模型的引入而自我提升。据我们所知,GDC是首个兼具高精度与实用效率的零样本扩散分类器。

原文摘要 · Abstract (English)

Large diffusion models have become mainstream generative models in both academic studies and industrial AIGC applications. Recently, a number of works further explored how to employ the power of large diffusion models as zero-shot classifiers. While recent zero-shot diffusion-based classifiers have made performance advancement on benchmark datasets, they still suffered badly from extremely slow classification speed (e.g., ~1000 seconds per classifying single image on ImageNet). The extremely slow classification speed strongly prohibits existing zero-shot diffusion-based classifiers from practical applications. In this paper, we propose an embarrassingly simple and efficient zero-shot Gaussian Diffusion Classifiers (GDC) via pretrained text-to-image diffusion models and DINOv2. The proposed GDC can not only significantly surpass previous zero-shot diffusion-based classifiers by over 10 points (61.40% - 71.44%) on ImageNet, but also accelerate more than 30000 times (1000 - 0.03 seconds) classifying a single image on ImageNet. Additionally, it provides probability interpretation of the results. Our extensive experiments further demonstrate that GDC can achieve highly competitive zero-shot classification performance over various datasets and can promisingly self-improve with stronger diffusion models. To the best of our knowledge, the proposed GDC is the first zero-shot diffusionbased classifier that exhibits both competitive accuracy and practical efficiency.

零样本分类扩散模型高效推理图像生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。