用文本生成零样本学习的判别性图像,提升分类性能。
Discriminative Image Generation with Diffusion Models for Zero-Shot Learning
- 通过判别性类别令牌生成高质量图像。
- 在4个数据集上超越无人工标注的方法,接近人工标注水平。
- 适合需要自动生成图像的零样本学习场景。
生成式零样本学习(ZSL)方法基于预定义的类别语义原型合成类别相关特征,表现优异。然而,这种特征生成范式缺乏可解释性,且依赖人工标注的语义原型,难以推广至通用场景。为克服这些缺陷,我们提出一种新方法:利用文本提示为未见类别生成图像。为此,本文提出DIG-ZSL框架,通过预训练的类别判别模型引导,为每个未见类别学习判别性类别令牌(DCT)。借助DCT,可生成多样且高质量的图像,作为有效的未见样本用于ZSL任务。在四个数据集上的大量实验与可视化表明,DIG-ZSL能够生成高质量、多样化的图像,在非人工标注原型方法中显著优于现有最先进水平,并达到或超过依赖人工标注原型基线的性能。代码将在论文录用后公开。
原文摘要 · Abstract (English)
Generative Zero-Shot Learning (ZSL) methods synthesize class-related features based on predefined class semantic prototypes, showcasing superior performance. However, this feature generation paradigm falls short of providing interpretable insights. In addition, existing approaches rely on semantic prototypes annotated by human experts, which exhibit a significant limitation in their scalability to generalized scenes. To overcome these deficiencies, a natural solution is to generate images for unseen classes using text prompts. To this end, We present DIG-ZSL, a novel Discriminative Image Generation framework for Zero-Shot Learning. Specifically, to ensure the generation of discriminative images for training an effective ZSL classifier, we learn a discriminative class token (DCT) for each unseen class under the guidance of a pre-trained category discrimination model (CDM). Harnessing DCTs, we can generate diverse and high-quality images, which serve as informative unseen samples for ZSL tasks. In this paper, the extensive experiments and visualizations on four datasets show that our DIG-ZSL: (1) generates diverse and high-quality images, (2) outperforms previous state-of-the-art nonhuman-annotated semantic prototype-based methods by a large margin, and (3) achieves comparable or better performance than baselines that leverage human-annotated semantic prototypes. The codes will be made available upon acceptance of the paper.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。